Radiotherapy systems and related methods

Abstract

A method of establishing a biomechanical model (300) for an anatomical region (202) of a radiotherapy patient (200) in a radiotherapy system (100) is disclosed. The method comprises the following steps: establishing a point-based or grid-based model of the modeled anatomical region; associating static and / or dynamic features with points or mesh nodes (302) of the point-based or mesh-based model, the features comprising world space coordinates of the points or mesh nodes and state variables representing spatio-temporal displacements of the anatomical region, and for each point or mesh node, characterizing at least one feature of body tissue in the anatomical region; representing the model as a graph in a graph neural network system (500); and training the model on the cyclic or semi-cyclic motion of the anatomical region. The invention also discloses a radiotherapy system.

Description

Technical Field

[0001] The present disclosure relates to a radiotherapy system including a control system for controlling an ionizing radiation beam supervision system. In particular, the present disclosure relates to predicting movement of an anatomical region, particularly cyclic or semi-cyclic movement, to improve the accuracy of dose delivery. Background Art

[0002] Treatment of disease, such as cancer, using external beam radiation therapy involves applying ionizing radiation to a patient so that the radiation energy is deposited in malignant cells of the patient's body. If sufficient amounts of energy are deposited, it may result in DNA damage and subsequent death of the irradiated cells.

[0003] Ionizing radiation can be direct or indirect ionizing. Direct ionizing radiation utilizes charged particles, such as electrons, protons, alpha particles, and heavy ions. Indirect ionizing radiation utilizes neutral particles, such as photons (X-rays and gamma rays) and neutrons. The present disclosure is applicable to both types of external beam radiation therapy, i.e., using direct or indirect ionizing radiation. In particular, the present disclosure is applicable to X-ray or proton radiation therapy.

[0004] Protons and other charged particles display depth-dose curves suitable for radiation therapy. Because of their relatively large mass, protons and other charged particles have little lateral scatter in tissue, so particle beams can be focused on malignant tissue, minimizing dose side effects to surrounding healthy tissue. Particle beams can also use the Bragg peak effect (i.e., the tendency of charged particles to deposit most of their energy in the final region of the particle beam trajectory) to more accurately target malignant tissue.

[0005] US2016 / 0144201A1 discloses a method for creating a proton therapy plan, the method comprising the steps of dividing a volume of interest into sub-volumes, applying dose constraints to the sub-volumes based on, inter alia, patient motion, finding one or more feasible configurations of a proton therapy system, and selecting a proton beam configuration that improves or optimizes one or more aspects of the proton therapy.

[0006] However, a problem associated with the proton therapy system of US2016 / 0144201A1 and other prior art external beam radiotherapy systems is that patient movement during treatment makes it difficult for the system to accurately deliver the radiation dose to the intended location within the radiation target. In fact, even if the patient is restrained, organ movement within the patient's body may still make it difficult for prior art radiotherapy systems to accurately deliver the radiation dose, as does motion caused by breathing and involuntary muscle activity (e.g., heartbeat).

[0007] Therefore, when preparing a treatment plan using a prior art system, the uncertainty associated with patient and / or organ movement during treatment needs to be taken into account when planning dose delivery. In practice, this uncertainty may cause the operator of the system to set the system to deliver a lower total dose to avoid damaging healthy tissue. This in turn may result in a less effective treatment than if the uncertainty did not exist.

[0008] In particular, in current radiotherapy practice, relatively large margins are added to the tumor volume in the thoracic and upper abdominal cavities to compensate for the effects of respiratory motion on tumor dose delivery. This can lead to compromises in the prescribed tumor dose as well as treatment planning, which can adversely affect treatment outcomes and increase radiation-induced morbidity.

[0009] WO2014 / 116868A1 discloses a motion compensation system for tracking and compensating patient motion during a medical imaging scan. The system includes an optical marker and a first and a second optical detector, the optical marker including an optically visible pattern, the first and the second optical detector being positioned to digitally image the optically visible pattern along a first line of sight and a second line of sight, respectively. The system further includes: a tracking engine configured to determine the posture of an object in six degrees of freedom by analyzing images from the first optical detector and the second optical detector; and a controller interface configured to generate tracking information based on the posture and electronically transmit the tracking information to a scanner controller to enable compensation for object motion within a medical imaging scanner. WO2014 / 116868A1 suggests that the disclosed method for tracking patient motion can be applied in a therapeutic environment, for example, to track patient motion caused, for example, by breathing, so as to direct a therapeutic radiation beam to a diseased tissue region while avoiding surrounding healthy tissue.

[0010] However, while the system disclosed in WO 2014 / 116868 A1 may be able to track patient movement to some extent by virtue of the system tracking optical markers and assigning movement of the optical markers to movement of the patient, the system does not take into account dynamic deformation or warping of the tissue region to which the optical markers are attached.

[0011] Biomechanical models for predicting patient motion are known in the art. Such biomechanical models are discussed, for example, in the following articles, “Patient-specific biomechanical model for predicting lung motion from 4D CT images” (2014), Fuerst et al., IEEE Transactions on Medical Imaging 34.10.1109 / TMI.2014.2363611 and “Sensitivity of tumor motion simulation accuracy to lung biomechanical modeling methods and parameters” (2015), Tehrani et al., Phys Med Biol. 2015 Nov 21; 60(22): 8833-49. doi: 10.1088 / 0031-9155 / 60 / 22 / 8833. Epub 2015 Nov 4. PMID: 26531324; PMCID: PMC4652597. However, known biomechanical models, such as those discussed in the above-mentioned articles, are computationally time-consuming and unsuitable for real-time use during radiation therapy dose delivery. Summary of the invention

[0012] In view of the above problems, an object of the present disclosure is to provide a radiation therapy system and related systems and methods that can at least alleviate the above problems and / or reduce the uncertainty of target positioning during dose delivery.

[0013] Another object of the present disclosure is to provide a radiation therapy method and system involving a beam supervision system that allows for accurate dose delivery to an irradiation target.

[0014] Another object of the present disclosure is to provide a radiation therapy system including a beam supervision system, wherein the beam supervision system can accurately track an irradiation target.

[0015] Yet another object of the present disclosure is to provide a radiotherapy system in which irradiation of risk organs can be avoided.

[0016] It is a further object of the present disclosure to provide a radiotherapy system which allows at least partially compensating for delays of a beam supervision system.

[0017] It is yet another object of the present disclosure to provide a method of building a model of an anatomical region of a patient for radiation treatment in a radiation treatment system.

[0018] Furthermore, it is an object of the present disclosure to provide a computer-implemented method for planning at least a portion of a radiation therapy procedure to be performed on an anatomical region of a radiation therapy patient.

[0019] The claimed invention is specified in the independent claims of the present application. Advantageous modifications and versions of the claimed invention are specified in the independent claims.

[0020] As previously mentioned, the motion of the anatomical region can be modeled using a biomechanical model. The partial differential equations underlying such a model can be solved in space and time using numerical computational methods, such as finite element analysis, finite difference or finite volume methods, etc. The calculations can be performed iteratively using a time-stepping scheme that allows the evolution of the model state to be obtained. However, as previously mentioned, such calculations are slow and require a large amount of computing resources and are therefore not suitable for real-time use during radiotherapy dose delivery.

[0021] In the paper “Learning to Simulate Complex Physics with Graph Networks,” Sánchez-Gonzalez et al., arXiv:2002.09405v2[cs.LG]14.09.2020 (available at https: / / arxiv.org / abs / 2002.09405), we discuss and establish that graph network-based simulators (GNSs) are capable of accurately simulating a wide range of physical systems where fluids, rigid solids, and deformable materials interact.

[0022] In the above-mentioned papers, a particle-based representation of physical systems is adopted, where each particle represents the state of the system. The physical dynamics are approximated by interactions between particles, for example, by exchanging energy and momentum between their neighbors. Particle-based simulation can be viewed as message passing on a graph. Nodes correspond to particles and edges correspond to pairwise relations between particles, based on which interactions are computed. The learnable simulator uses a parameterized function approximator d θ Compute dynamic information, whose parameters can be optimized for some training goal. Parameterized function approximator has three steps - encoder, processor, decoder.

[0023] The encoder embeds a particle-based state representation as an input graph, where node embeddings are learned functions of particle states, directed edges are added to create paths between particle nodes with some potential interactions, and graph-level embeddings can be added to represent global properties.

[0024] The processor computes the interactions between nodes via a learned message passing step to generate an output graph.

[0025] The decoder extracts dynamic information from the nodes of the output graph.

[0026] In the paper “Learning Grid-Based Simulations with Graph Networks”, Pfaff et al., arXiv:2010.03409v4[cs.LG]18.06.202 (available at https: / / arxiv.org / abs / 2010.03409) disclose a framework for learning grid-based simulations using a graph neural network that implements an encoder-processor-decoder architecture followed by an integrator.

[0027] In the discussed framework, a simulation mesh with nodes connected by mesh edges is used to describe the state of the system at time t. Each node is associated with a reference grid space coordinate spanning the simulation mesh and an additional dynamic quantity to be modeled. In systems where the mesh represents a moving and deforming surface or volume, in addition to the fixed grid space coordinates, the mesh contains additional world space coordinates that describe the dynamic state of the mesh in 3D space.

[0028] The framework learns a forward model of the dynamics of the grid at time t+1 given the current grid (i.e., the grid at time t).

[0029] The encoder encodes the current grid into a multigraph. Grid nodes become graph nodes, and grid edges become bidirectional grid edges in the graph. World edges are added to the graph to enable learning of external dynamics. World edges are edges created between nodes that are very close in space, excluding those pairs of nodes that are already connected by grid edges. World edges make it possible to model connectivity not only between adjacent grid nodes, but also between nodes that are spatially close in world space but far away in grid space. Features are encoded into graph nodes and edges.

[0030] The relative displacement vector is encoded in mesh space and its norm is encoded into the mesh edge. The relative world space displacement vector and its norm are encoded into both the mesh edge and the world edge. The remaining dynamic features are provided as node features along with a one-hot vector indicating the node type.

[0031] The processor consists of multiple identical message-passing blocks. Each block contains a separate set of network parameters and is applied sequentially to the output of the previous block to update mesh edges, world edges, and node embeddings.

[0032] After the final processing step of the processor, the latent node features are transformed into one or more output features in the decoder. The output features can be interpreted as (higher-order) derivatives of the dynamic features and can be integrated to calculate the next step dynamic quantity. Additional output features can be used to make direct predictions of auxiliary quantities.

[0033] Finally, the output mesh nodes are updated to produce the mesh at time t+1.

[0034] The present disclosure is based on the discovery that, if properly adopted and implemented, a graph network system can be used to model and predict in real time cyclic or semi-cyclic movements or motions, particularly respiratory movements or motions, of anatomical regions including irradiation targets during radiation therapy dose delivery.

[0035] Predicted cyclic or semi-cyclic movement or motion of an anatomical region can be used to control an ionizing radiation beam during a radiotherapy session, thereby achieving a desired dose delivery in the anatomical region. For example, the alignment of the beam can be controlled based on the predicted movement or motion, thereby continuously tracking the irradiation target. Additionally or alternatively, the beam can be blocked or turned off when the predicted movement or motion of the anatomical region indicates that the irradiation target will venture outside the range or trajectory of the ionizing radiation beam and / or when the predicted movement or motion indicates that the ionizing radiation beam will intersect with the risk organ. Therefore, the present disclosure is applicable to radiotherapy systems in which the alignment of the ionizing radiation beam can be controlled and to radiotherapy systems in which the alignment of the ionizing radiation beam cannot be controlled. In the first case, a beam tracking function and / or a beam blocking or beam shutting function can be used, and in the latter case, only a beam blocking or beam shutting function can be used.

[0036] According to a first aspect, the present disclosure provides a method for establishing a model of an anatomical region of a patient for radiation therapy in a radiation therapy system, the method comprising the following steps:

[0037] (a) establishing a point-based or grid-based representation or model of the anatomical region;

[0038] (b) associating static and / or dynamic features with points or mesh nodes of the point-based or mesh-based representation or model, the features comprising world-space coordinates of the points or mesh nodes and state variables representing spatiotemporal displacements of the anatomical region, and, for each point or mesh node, at least one feature characterizing body tissue in the anatomical region;

[0039] (c) representing point-based or grid-based representations or models as graphs in graph neural network systems; and

[0040] (d) Training point-based or mesh-based representations or models on cyclic or semi-cyclic motions of anatomical regions.

[0041] The step of establishing a point-based or mesh-based representation or model of the anatomical region typically includes modeling the anatomical region as interconnected points or as a mesh comprising interconnected mesh nodes, and the step of representing the point-based or mesh-based representation or model as a graph in a graph neural network system typically includes adopting a particle-based or mesh-based representation of the anatomical region such that each particle or mesh node represents a state of the system and such that the physical dynamics are approximated by interactions between the particles or mesh nodes.

[0042] The step of training a point-based or mesh-based representation or model on the cyclic or semi-cyclic motion of the anatomical region typically includes training a graph neural network system on the cyclic or semi-cyclic motion of the anatomical region, i.e., training the graph neural network system to predict the cyclic or semi-cyclic motion of the anatomical region.

[0043] The at least one feature characterizing the body tissue in the anatomical region may advantageously include at least one variable characterizing the interaction between the anatomical region and the ionizing radiation beam. In particular, the at least one feature characterizing the interaction between the anatomical region and the ionizing radiation beam may include at least one of the following: linear stopping power; relative stopping power; radiodensity and radiopacity.

[0044] The step of representing the model as a graph in a graph neural network system may include establishing graph nodes, graph edges, and graph global elements having characteristics reflecting static and dynamic variables of the biomechanical model.

[0045] The graph network system may include an input, a graph processing neural network, an updater, and an output, and the graph processing neural network may include an encoder, a processor, and a decoder.

[0046] The encoder may be configured to receive a current state of the grid-based model from an input and encode an input graph from the current state of the model. In addition, the processor may be configured to process the input graph and provide an output graph with updated node and edge embeddings.

[0047] Furthermore, the decoder may be configured to decode the output graph, extract output features corresponding to static and dynamic variables of the biomechanical model, and the updater may be configured to update the output features to produce an inferred future state of the mesh-based biomechanical model, and provide the inferred state to the output.

[0048] The step of training the model on cyclic or semi-cyclic motion of the anatomical region may include applying a loss function to the inferred state provided by the updater and to a real-world dataset on an anatomical region corresponding to the modeled anatomical region.

[0049] The real-world dataset may include at least one of 4D computed tomography (4D-CT) scans, 4D magnetic resonance imaging (4D-MRI), and ultrasound imaging.

[0050] The real-world dataset may include the evolution of images covering the modeled anatomical region during a respiratory cycle.

[0051] The point-based or grid-based model may incorporate at least one of a linear elastic model and a hyperelastic model adapted to the physical properties of the anatomical region.

[0052] The method may include the following steps:

[0053] - dividing the anatomical region into at least a first anatomical sub-region displaying a first set of physical properties and a second anatomical sub-region displaying a second set of physical properties different from the first set of physical properties;

[0054] - for a first anatomical sub-region, assigning to the point-based or grid-based model a first point-based or grid-based sub-model applicable to the first set of physical properties; and

[0055] - for a second anatomical sub-region, assigning to the point-based or mesh-based model a second point-based or mesh-based sub-model adapted for a second set of physical properties.

[0056] The first and second anatomical sub-regions may share a common boundary, and the method may comprise the step of assigning a common boundary condition to the first and second point-based or mesh-based sub-models in the common boundary.

[0057] Such a graph network system can be implemented according to either of the papers: “Learning to Simulate Complex Physics with Graph Networks”, Sanchez-Gonzalez et al., arXiv:2002.09405v2, and “Learning Grid-Based Simulations with Graph Networks”, Pfaff et al., arXiv:2010.03409v4.

[0058] According to a second aspect, the present disclosure provides a computer-implemented method of planning at least a portion of a radiation therapy procedure to be performed on an anatomical region of a radiation therapy patient, comprising:

[0059] - establishing a biomechanical model of the anatomical region (202) according to the method of the first aspect; and

[0060] - Each point or grid node is assigned at least one of: a planned dose and a maximum allowed dose.

[0061] According to a third aspect, the present disclosure provides a method of providing a radiotherapy system, the radiotherapy system comprising at least one beam source, the at least one beam source being arranged to irradiate an ionizing radiation beam onto an irradiation target located in an anatomical region of a radiotherapy patient. The method comprises the following steps:

[0062] -Build biomechanical models of anatomical regions;

[0063] - Train the model on cyclic or semi-cyclic motions of anatomical regions;

[0064] - Implementing the training model in a graph network system implemented in the control system of the radiotherapy system;

[0065] - in a graph network system, establishing a current state of the training model based on data regarding a current position and / or a current orientation of the anatomical region in space;

[0066] - in the graph network system and based on the current state of the trained model, establishing an inferred future state of the trained model; and

[0067] - controlling an ionizing radiation beam of at least one beam source based on the inferred future state.

[0068] The step of controlling the ionizing radiation beam of the at least one beam source may include controlling the alignment of the at least one beam source such that the ionizing radiation beam tracks the irradiation target based on the inferred future state.

[0069] Additionally or alternatively, the step of controlling the ionizing radiation beam of at least one beam source may comprise blocking the ionizing radiation beam or switching off at least one beam source if the inferred future state indicates that the irradiation target will venture outside the trajectory of the ionizing radiation beam.

[0070] Additionally or alternatively, the step of controlling the ionizing radiation beam of at least one beam source may comprise blocking the ionizing radiation beam or shutting down at least one beam source if the inferred future state indicates that the trajectory of the ionizing radiation beam is at risk of intersecting the organ at risk.

[0071] Blocking the radiation beam may be achieved by applying a gating function to a beam supervision system, ie to a subsystem of the radiotherapy system controlling at least one beam source.

[0072] The cyclic or semi-cyclic movement may generally be a breathing movement.

[0073] The step of establishing the biomechanical model may include establishing a point-based or mesh-based model of the anatomical region and associating static and / or dynamic features with the points or mesh nodes of the point-based or mesh-based model. The features may include world space coordinates of the points or mesh nodes and state variables representing the spatiotemporal displacement of the anatomical region. The state variables may be a displacement field, i.e., an assignment of displacement vectors of all points or mesh nodes in the anatomical region that are displaced from one state to another.

[0074] The step of implementing the training model in the graph network system may include representing the training model as a graph in the graph network system by establishing graph nodes, graph edges, and graph global elements having characteristics that reflect static and dynamic variables of the training model.

[0075] The step of establishing the current state of the training model may comprise associating the current state with data regarding the current position and / or orientation of the anatomical region in space provided by a monitoring system of the radiation therapy system configured to monitor the anatomical region.

[0076] The monitoring system may be configured to monitor at least one of: a position and / or orientation in space of a boundary region of the anatomical region; and a position and / or orientation in space of a fiducial marker in the anatomical region.

[0077] The variables may include at least one variable characterizing the interaction between the anatomical region and the ionizing radiation beam, for example, linear stopping power and / or radiopacity. The variables may additionally or alternatively include at least one variable characterizing at least one of: a planned radiation dose; a maximum allowed radiation dose; and a cumulative radiation dose.

[0078] The step of providing an inferred future state of the trained model may include updating a variable characterizing the cumulative radiation dose, and the step of controlling the position and / or alignment of at least one beam source and / or blocking the ionizing radiation beam may include instructing a beam supervision system of the radiotherapy system to position and align at least one beam source so that the irradiated target receives the planned radiation dose without exceeding the maximum allowable radiation dose to other parts of the anatomical region.

[0079] According to a fourth aspect, the present disclosure provides a radiotherapy system, comprising:

[0080] - at least one beam source arranged to provide a beam of ionizing radiation to an irradiation target located in the anatomical region;

[0081] - a beam supervision system configured to control at least one beam source;

[0082] - a monitoring system arranged to monitor the anatomical region and optionally a beam supervision system;

[0083] - a control system comprising a computer-implemented graph network system in which a model of an anatomical region trained on cyclic or semi-cyclic motions of the anatomical region is implemented;

[0084] - the graph network system is configured to establish a current state of the training model based on data received from the monitoring system regarding a current position and / or a current orientation of the anatomical region in space, and is configured to establish an inferred future state of the training model based on the current state; and

[0085] - The control system is configured to instruct the beam supervision system to control the ionizing radiation beam of the at least one beam source based on the inferred future state.

[0086] The control system may be configured to instruct the beam supervision system to control the ionizing radiation beam of the at least one beam source based on the inferred future state so that the at least one beam source tracks the irradiation target.

[0087] Alternatively or additionally, the control system is configured to instruct the beam supervision system to control the ionizing radiation beam of at least one beam source based on the inferred future state to block the ionizing radiation beam of at least one beam source or shut down at least one beam source if the inferred future state indicates that the irradiation target will venture outside the trajectory of the ionizing radiation beam.

[0088] Alternatively or additionally, the control system is configured to instruct the beam supervision system to control the ionizing radiation beam of at least one beam source based on the inferred future state to block the ionizing radiation beam of at least one beam source or shut down at least one beam source if the inferred future state indicates that the trajectory of the ionizing radiation beam is at risk of intersecting the organ at risk.

[0089] The training model can be a point-based or grid-based model, including static and / or dynamic features associated with points or grid nodes of the point-based or grid-based model, the features including world space coordinates of the points or grid nodes and state variables representing the spatiotemporal displacement of the anatomical region.

[0090] The graph network system may be configured to represent the training model as a graph including graph nodes, graph edges, and graph global elements having characteristics reflecting static and dynamic variables of the training model.

[0091] The graph network system may be configured to associate the current state of the build of the training model with the data received from the monitoring system.

[0092] The monitoring system may be configured to monitor at least one of: a position and / or orientation in space of a boundary region of the anatomical region; and a position and / or orientation in space of a fiducial marker in the anatomical region.

[0093] The variables may include at least one variable characterizing an interaction between the anatomical region and the ionizing radiation beam, and at least one variable characterizing at least one of: a planned radiation dose; a maximum allowed radiation dose; and a cumulative radiation dose. For each inferred future state of the training model, the graph network system may be configured to update the variable characterizing the cumulative radiation dose, and the control system may be configured to instruct the beam supervision system to control the position and / or alignment of at least one beam source so that the irradiation target receives the planned radiation dose without exceeding the maximum allowed radiation dose of other portions of the anatomical region.

[0094] According to a fifth aspect, the present disclosure provides a radiotherapy system, comprising:

[0095] - at least one beam source arranged to deliver a beam of ionizing radiation to an anatomical region comprising an irradiation target;

[0096] - a beam supervision system configured to control the position and / or alignment of at least one beam source;

[0097] - a monitoring system arranged to provide updated current data about the position and / or orientation of the anatomical region in space;

[0098] - a control system comprising a computer-implemented graph network system, wherein a model of an anatomical region trained on cyclic or semi-cyclic motions of the anatomical region is represented as a graph comprising nodes, edges and global elements and comprising static attributes reflecting static characteristics of the anatomical region and dynamic attributes reflecting dynamic characteristics of the anatomical region, the control system being configured to:

[0099] - providing a current representation or status of the anatomical region based on information received from the monitoring system;

[0100] - processing a current representation or state in the graph network system to provide at least one inferred future representation or state of the anatomical region; and

[0101] - controlling an ionizing radiation beam of at least one beam source based on at least one inferred future representation or state of the anatomical region.

[0102] Thus, the radiotherapy system provides the control system with predicted future values ​​of the dynamic characteristics of the anatomical region, thereby allowing the beam supervision system to employ the alignment of the beam to take into account predicted movements and deformations of the anatomical region, in particular movements and deformations of the anatomical region due to the respiratory activity of the radiotherapy patient. This allows the radiotherapy system to accurately track the irradiation target while taking due account of surrounding tissue, in particular organs at risk. This also allows at least partial compensation of delays of the beam supervision system, in particular delays of the beam guidance subsystem of the beam supervision system.

[0103] Additionally or alternatively, the control system may be configured to block the ionizing radiation beam or shut down at least one beam source if the predicted future value of the dynamic characteristic of the anatomical region indicates that the ionizing radiation beam is likely to intersect the organ at risk.

[0104] Additionally or alternatively, the control system can be configured to block the ionizing radiation beam or shut down at least one beam source if predicted future values ​​of dynamic characteristics of the anatomical region indicate that the irradiation target will venture outside the trajectory of the ionizing radiation beam, for example, due to movement of the anatomical region bringing the irradiation target outside the range of at least one beam source.

[0105] The map network system may preferably be configured to repeatedly provide at least one extrapolated future representation or state of the anatomical region during dose delivery, preferably with a period in the range of 20 to 100 ms, for example 50 ms.

[0106] A graph network system may include: an encoder configured to encode an input graph from a current representation of an anatomical region; a processor comprising one or more graph network blocks and trained to process the input graph to perform message passing along graph edges and provide an output graph with updated node and edge embeddings; and a decoder configured to decode the output graph and extract at least one output feature from the output graph.

[0107] Such a graph network system can be implemented according to either of the papers: “Learning to Simulate Complex Physics with Graph Networks”, Sanchez-Gonzalez et al., arXiv:2002.09405v2, and “Learning Grid-Based Simulations with Graph Networks”, Pfaff et al., arXiv:2010.03409v4.

[0108] The features may advantageously include features characterizing body tissue in the anatomical region. Tissue properties may include linear or relative stopping power and / or radiodensity or radiopacity.

[0109] The characteristics may also include at least one of a planned dose, a cumulative dose, and a maximum allowed dose associated with each point or grid node.

[0110] Information about the position and / or alignment of the beam source may be provided to the control system. In addition to being based on information received from the imaging system, the current representation of the anatomical region may be based on information about the alignment of the beam source. Then, if the dynamic feature includes absorbed dose, the inferred future representation of the anatomical region will also be updated with respect to the spatial distribution of absorbed dose, allowing control of dose delivery and beam alignment based on the difference between the planned dose and the absorbed dose.

[0111] The representation of the anatomical region may be provided as a mesh comprising mesh nodes and mesh edges, in which case the static and dynamic properties may reflect the static and dynamic characteristics of the mesh nodes.

[0112] According to a sixth aspect, the present disclosure provides a computer-implemented control system for controlling an ionizing radiation beam supervision system to track an irradiation target in an anatomical region of a radiotherapy patient during dose delivery and / or to block the ionizing radiation beam if the ionizing radiation beam is at risk of intersecting an organ at risk or if the irradiation target is about to venture out of the track of the ionizing radiation beam, the control system comprising a computer-implemented neural network system for processing data representing the irradiation target, wherein the anatomical region (including the irradiation target) is represented in the neural network system using a grid of edges E M Simulated mesh M of connected nodes V t =(V, E M ) model, where each node v i ∈V is associated with:

[0113] - an input configured to receive the current state M of the simulation grid t and optionally at least one previous state M t-1 ;

[0114] - The reference grid space coordinates u across the simulation grid i ;

[0115] - at least one dynamic quantity q i ,as well as

[0116] - The world space coordinate x that describes the dynamic state of the simulation mesh in 3D space i ,

[0117] Among them, the neural network system includes:

[0118] -Graph processing neural networks;

[0119] - Updater; and

[0120] - an output configured to provide at least one inferred dynamic state M of the simulation grid to the beam supervision system t+1 ,

[0121] Among them, graph processing neural networks include:

[0122] - an encoder configured to generate a t The input graph G 0 Encoding;

[0123] - a processor, which includes one or more graph network blocks and is trained to process an input graph G 0 to perform message passing along grid edges and world edges and provide an output graph G with updated node and edge embeddings M ;as well as

[0124] - a decoder configured to decode the output graph G M , extract at least one dynamic quantity q i Corresponding to at least one output feature p i ;

[0125] The updater is configured to update at least one dynamic quantity q i t+1 ＝ p i+ q i t , to generate at least one inferred dynamic state M t+1 .

[0126] According to a seventh aspect, the present disclosure provides a computer-implemented method for modeling cyclic or semi-cyclic motion of an anatomical region including an irradiation target, comprising the following steps:

[0127] - Provide a mesh edge E that describes the anatomical region M Simulated mesh M of connected nodes V t =(V, E M ), where each node vi ∈V and spans the simulation grid M t The reference grid space coordinate u i , describing the simulation grid M in 3D space t The world space coordinate x of the dynamic state i and optionally at least one additional dynamic quantity q i associated;

[0128] - For the nodes that are not yet connected in the simulation grid and have a value less than the predetermined value |x i -x j | <r w For every pair of nodes in spatial proximity, create a world edge E W ;

[0129] - In the encoder of the graph network system, by shifting the relative grid space vector u ij =u i -u j and its norm |u ij |Encoded to each grid edge e ij M ∈E M and the relative world space displacement vector x ij =x i -x j and its norm |x ij |Encoded to each grid edge e ij M ∈E M Neutralize each world edge ij M ∈E W In the simulation, the current simulation grid M t Encoding to multi-graph G = (V, E M , E W ), and for each node v i ∈V provides the optional at least one additional dynamic quantity (q i ) as node features;

[0130] - in a processor of a graph network system comprising a plurality of identical message passing graph network blocks, updating each mesh edge, each world edge and each node embedding

[0131]

[0132] By computing the interactions between nodes via a predetermined number (M) of learning message passing steps, an updated latent graph sequence G = (G1, ..., G M ), where G m+1 =GN m+1 (G m), and returns the final updated graph G M =Processor(G 0 );

[0133] -In the decoder of the graph network system, the node feature v i Transformed into one or more output features p i ;as well as

[0134] -In the updater of the graph network system, by updating the output feature q i t+1 =p i +q i t Integrate to calculate the next step dynamic characteristic q i t+1 , and use the next step dynamic feature q i t+1 Update the output grid nodes to produce at least one inferred dynamic state M t+1 .

[0135] According to an eighth aspect, the present disclosure provides a biomechanical model of an anatomical region including an irradiation target, the model being trained in a trained graph network system for cyclic or semi-cyclic motion.

[0136] Features of any of the above aspects may be combined with features of any of the other aspects.

[0137] The disclosed systems and methods involve an anatomical biomechanical model that represents and simulates a patient's internal state, and a graph neural network trained on the simulated biomechanical data to learn the patient's internal dynamics.

[0138] Biomechanical models can be used to simulate the movement and deformation of internal organs during breathing, which can be used to predict the internal state of a patient during radiation therapy or surgical procedures.

[0139] Biomechanical models can rely on a collection of three-dimensional (3D) meshes in space, each representing a single anatomical structure, or on a collection of point clouds that define various physical domains (meshless), such as organs and tissues.

[0140] The biomechanical model can be mathematically defined by a set of dynamic and constitutive equations, as well as boundary conditions, parameters, and initial values.

[0141] The biomechanical model can be solved in space and time to produce a series of estimates of the 3D deformation field, representing the dynamic internal motion of the anatomical structure, and can be used to predict the location of radiation deposition.

[0142] Biomechanical models can be solved in space and time using numerical computational methods such as finite element analysis, finite difference or finite volume methods.

[0143] The calculations can be performed iteratively using a time-stepping scheme that allows obtaining a near real-time evolution of the model state during radiation delivery. Here, real-time means updating the model state at a high enough cadence to allow changes in the internal anatomy to be captured and recorded with sufficient resolution.

[0144] High-resolution simulations are typically very slow, and according to the present disclosure, alternative learned models can be used to provide faster predictions. Biomechanical modeling (mesh-based or particle-based) can be used to generate data to train a neural network, which can then be used for real-time calculations of patient tissue biomechanics.

[0145] The computational model can predict the 3D spatial coordinates of the target tissue and the tissue at risk, as well as spatially varying tissue parameters such as Hounsfield units (HU) and / or relative stopping power (RSP) at a future requested time step t, which can be in the range of milliseconds to seconds. Prior to simulation, spatially varying quantities such as HU and RSP can be extracted from the patient reference CT image and assigned to corresponding nodes in the 3D anatomical model. Since the spatial resolution of the mesh or point cloud may be lower than that of the original CT image, each data node can represent a neighborhood of values.

[0146] In this article, the term "model" refers to a model that is capable of simulating (and predicting) displacements in space and time given a set of input parameters and a description of the system's dynamics. The term "3D anatomical model" may refer to a geometric representation of an anatomical structure or an entire patient. The anatomical region may be the chest, abdomen, head and neck, or brain. The anatomical structure may be a solid cancerous or non-cancerous tumor, organ, tissue, or a combination thereof. The proposed motion prediction method is applicable to any structure that undergoes cyclic or semi-cyclic translation, rotation, and / or deformation.

[0147] Biomechanical modeling

[0148] The dynamic model of the physical domain can be broadly defined by a set of partial differential equations f with source terms f, boundary conditions g and h, and model parameters p. The model can also include a set of constitutive equations and intermediate variables. The model state variables u can be used to represent the space-time displacement of the domain Ω. t represents the time dimension.

[0149]

[0150] In the disclosed systems and methods, the state variable u can advantageously be a displacement field, i.e., an assignment of displacement vectors for all points in a domain that are displaced from one state to another. A displacement vector specifies the position of a point or particle relative to an origin or previous position. Thus, in the disclosed systems and methods, a displacement field can be used to describe the effects of deformation on an anatomical region, including an irradiation target.

[0151] The mathematical model above is defined in such a way that it reproduces the relevant dynamics of the physical domain within a certain range of accuracy (in this case, quantized as u) and is constrained by a set of boundary conditions and model parameters representing the physical properties and latent variables.

[0152] Each individual physical domain can be modeled by a different set of equations, and therefore the construction of such a model may require expert knowledge of the physics and mechanics of the respective domain. The requirement is that the model is able to produce spatially and temporally resolved displacements of the domain nodes. The same boundary conditions can be applied to domains that share boundaries.

[0153] The biomechanical model may be based on the mathematical model discussed above or on any dynamic model formulated as a forward mapping integrating the state variable u from time t to t+1.

[0154] Elasticity Model

[0155] There are many mathematical models that describe the behavior of materials, including:

[0156] -Linear elastic model: Linear elasticity describes the deformation of a body under external or internal loads as being linearly proportional to the load. Hooke's law represents the behavior of the material and relates the unknown stresses and strains.

[0157] -Hyperelastic models: These materials not only exhibit nonlinear material behavior, but also derive large deformations and stress-strain relations from the strain energy density function. Examples of hyperelastic material models are the Neo-Hookean and Mooney-Rivlin models.

[0158] The biomechanical model may be combined with any of these mathematical models.

[0159] Grid-based and gridless methods

[0160] Biomechanical 3D anatomical models can be represented as tetrahedral meshes or point / particle clouds, for which different numerical methods can be used to solve the underlying partial differential equation systems.

[0161] Meshless / particle-based methods

[0162] Meshless methods can exploit unstructured point clouds for spatial discretization. The collocation of nodes within a domain can be arbitrary, or follow certain rules that ensure, for example, that a portion of the domain contains a higher density of nodes, allowing for higher accuracy of the solution in that subdomain.

[0163] Grid-based methods

[0164] Numerical simulations in biomechanics are often solved by the finite element method (FEM). The domain of an individual organ / tissue can be discretized into a finite set of interconnected nodes and cells, on which the solution of the finite element analysis can be calculated. Meshes can be generated from organ and tumor segmentation to divide the structure into small connected tetrahedral elements.

[0165] domain

[0166] The domain can be thought of as a physical continuum that can be described by the same set of equations, parameters, and constraints. The physical domain of a biomechanical model can be the anatomical structure of interest, which typically includes the tumor or tumors to be treated, the organ in which the tumor is formed, and the surrounding organs at risk (OARs). They can also include non-OARs and the outer surface of the body.

[0167] The domain can be divided into a finite number of elements or nodes, resulting in a mesh or point cloud that can be used as the basis for model discretization in schemes for numerically solving biomechanical models.

[0168] Boundary conditions

[0169] Boundary conditions can be defined prior to simulation in order to constrain the biomechanical model to a unique solution. In the case of a grid-based model, boundary conditions can be defined on small planes of each domain (boundary). Boundary conditions can specify the expected value of the state variable on the boundary (Dirichlet boundary conditions), or the gradient of the state variable (Neumann boundary conditions), or a combination of the two (Robin, mixed and Cauchy boundary conditions). Since the state variable is a displacement, which is a vector, the boundary condition is also a vector. The boundary conditions can then be obtained by calculating the deformation from one image (reference image) to another image (moving image). Therefore, in order to obtain the boundary conditions from the images, at least two images of the patient from the same imaging study taken at different times are required.

[0170] Boundary conditions may also be obtained from alternative data sources other than imaging, for example, from monitoring sensors, in which case the boundary conditions will typically be used to constrain sub-regions of the boundary.

[0171] The boundary conditions may be patient-specific, ie, they may be derived from the patient's own imaging data, or they may be derived from data of other patients.

[0172] Internal Force

[0173] By including force terms (also called source terms) in the differential equations, forces and pressures on internal organs driven by, for example, muscle movement, breathing, bolus and bladder filling, can be embedded in the biomechanical model. These forces can be used to impose further constraints on the model so that it behaves in a physiological (and patient-specific) manner and can therefore be tailored to the application.

[0174] Solving biomechanical models

[0175] The biomechanical model may be solved using any suitable FEM software.

[0176] Graph Network Framework

[0177] Methods and systems according to the present disclosure use a graph network framework or system to predict future states of a biomechanical model, for example, a framework based on the teachings of the paper Relational Inductive Biases, Deep Learning, and Graph Networks, Battaglia et al., arXiv:1806.01261v3[cs.LG] 17.10.2018 (available at https: / / arxiv.org / abs / 1806.01261).

[0178] Methods and systems according to the present disclosure also use a graph network framework or system to learn the dynamics of a biomechanical model.

[0179] In a graph network framework or system, a graph can generally be defined as a 3-tuple G = (g, V, E), where entities are represented by graph nodes V, relationships are represented by graph edges E, and system-level properties are represented by global properties g.

[0180] The main computational unit in a graph network framework or system is the graph network block (GN block). A graph network block is a "graph-to-graph" module that takes a graph as input, performs computation on the structure, and returns a graph as output.

[0181] The Graph Network block contains three sub-functions - one for each of the edges E, nodes V, and global attributes g. The sub-functions can be implemented by potentially using any class of neural networks. Each of these functions can be implemented using, for example, a multi-layer perceptron (MLP), a recurrent neural network (RNN), a long short-term memory (LSTM) recurrent neural network, or a convolutional neural network (CNN).

[0182] The implementation of the subfunction can be based on DeepMind's Sonnet library (https: / / github.com / deepmind / sonnet).

[0183] If the 3D biomechanical model is based on a mesh, the nodes and edges of the graph can be specified to represent the nodes and vertices of the mesh, respectively. The dynamic quantities of the system, i.e., the state variables and their derivatives, can be embedded in the edges of the graph. The processor of the graph network block takes the input graph and updates the dynamic quantities defined on the edges to produce an output graph.

[0184] The features received by each edge determine the physical interactions and properties from the biomechanical model encoded in the network.

[0185] The graph edges embed the relative displacement field u and its norm |u|. In addition, the internal forces in the model can be encoded as directed edge features between two adjacent nodes, representing the force exerted by one node on its neighbor node.

[0186] The differential operator can be approximated by message passing between neighboring nodes, which act as sub-functions (f M ) are embedded in the edges of the graph. These functions take as input edge features (e.g. derivatives and forces) and sender and receiver node features (coordinates, material properties, etc.) and output higher-order derivatives of the nodes. These in turn are used to integrate the dynamic variable x.

[0187] Graph nodes can also embed functions (f V ), independent of messages passed between graph nodes, e.g., transformations of material properties.

[0188]

[0189] Here, e ij M and e ij W The mesh and the world edge respectively.

[0190] <![CDATA[Input e ij M > <![CDATA[Input e ij W > <![CDATA[Input v i > <![CDATA[Output p i > <![CDATA[u ij ,|u ij |,x ij ,|x ij |]]> <![CDATA[x ij ,|x ij |]]> <![CDATA[n i ]]> <![CDATA[F i ,x' i ]]>

[0191] The following table describes features of a diagram of a system and method according to one aspect of the present disclosure.

[0192]

[0193]

[0194] For the first-order model, the dynamic quantity x at time t+1 is obtained by integrating the derivative at time t according to the following formula i :

[0195] x i t+1 =x' i +x i t

[0196] Therefore, for the first-order model, the dynamic quantity x is updated using the first-order derivative output by the graph network i .

[0197] For the second-order model, the dynamic quantity x at time t+1 is obtained according to the following formula i :

[0198] x i t+1 =x” i +2x i t -x i t-1

[0199] For the second-order model, x' i Embed as node features and use x" i For dynamic quantity x i Perform integration.

[0200] Hereinafter, the present invention will be discussed in more detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0201] The following drawings and the related descriptions are provided to illustrate embodiments of the present disclosure and not to limit the scope of the claims.

[0202] Figure 1 and Figure 2 An embodiment of a radiation therapy system according to the present disclosure is schematically shown.

[0203] Figure 3 The anatomical regions of the patient being treated with radiation therapy are shown.

[0204] Figure 4 Shown according to Figure 3 Mesh-based model of the anatomical region.

[0205] Figure 5 Shown according to Figure 4 Graph encoded from a grid-based model.

[0206] Figure 6 A diagram illustrating a network system.

[0207] Figure 7 A graph network block is shown.

[0208] Figure 8 Embodiments of a method of creating a biomechanical model of an anatomical region of a radiation therapy patient for use in a radiation therapy system are shown.

[0209] Fig. 9Embodiments of a computer-implemented method of planning at least a portion of a radiation therapy procedure to be performed on an anatomical region of a radiation therapy patient are shown. DETAILED DESCRIPTION

[0210] Embodiments of a radiation therapy system according to the present disclosure and how such a system may be provided will be discussed in more detail below in the context of a radiation therapy system 100 configured to predict and compensate for respiratory motion of a radiation therapy patient during dose delivery. However, it should be understood that the following teachings may be applied to predict and compensate for any type of cyclic or semi-cyclic motion.

[0211] Figure 1 and Figure 2 An embodiment of a radiation therapy system 100 according to the present disclosure is schematically illustrated. The radiation therapy system 100 is configured to irradiate an irradiation target in an anatomical region of a radiation therapy patient. Figure 3 FIG. 2 is an illustration of a radiation therapy patient 200 having an anatomical region 202 (in this case, the chest) in which an irradiation target 204 is present. Figure 1 and Figure 2 , the anatomical region and the irradiation target are schematically indicated by reference numerals 202 and 204, respectively.

[0212] The anatomical region 202 includes different anatomical structures with different tissue properties, some of which may be organs at risk OARs, ie structures or organs that preferably should not be irradiated during radiation therapy. The irradiation target 204 may typically be a tumor.

[0213] Providing the radiation therapy system 100 involves establishing a 3D mesh-based biomechanical model of the anatomical region 202 including mesh nodes and mesh vertices prior to a radiation therapy session, and associating each mesh node with static and dynamic variables characterizing properties of the anatomical region 202 in or at the mesh node, thereby allowing the state of the anatomical region 202 to be represented by a set of mesh nodes that encode properties within a local spatial region of the anatomical region 202.

[0214] Figure 4 Shows Figure 3A biomechanical model 300 of an anatomical region 202 is disclosed in the specification, comprising mesh nodes 302 and mesh vertices 304, wherein each mesh node 302 is associated with a plurality of static and dynamic variables. The static and dynamic variables include a world space coordinate x of the mesh node 302. The static and dynamic variables also include a state variable u representing a spatiotemporal displacement of the anatomical region 202. The state variable u can advantageously be a displacement field, i.e., an assignment of displacement vectors of all mesh nodes 302 in the anatomical region 202 that are displaced from one state to another. Typically, a displacement vector specifies the position of a point or particle relative to an origin or a previous position. Thus, the displacement field u can be used to describe the effect of deformation on the anatomical region, and therefore also on the irradiation target.

[0215] Advantageously, the static and dynamic variables may also include at least one variable characterizing the interaction between the anatomical region 202 and the ionizing radiation beam in or at each grid node 302 or voxel, such as linear stopping power or radiopacity, for example, expressed as Hounsfield units. The static and dynamic variables may also include variables characterizing the planned and / or maximum allowed radiation dose in or at each grid node 302. The static and dynamic variables may also include variables characterizing the cumulative radiation dose in or at each grid node 302.

[0216] The biomechanical model 300 can in principle be based on any mathematical model of the anatomical region that allows the movement and deformation of organs and tissues in the anatomical region 202 to be simulated. The biomechanical model can be mathematically defined by a set of dynamic and constitutive partial differential equations, as well as boundary conditions, parameters and initial values, which can be solved in space and time to produce a series of estimates of the 3D deformation field representing the dynamic internal motion of the anatomical region 202. Typically, such a solution involves solving the equations in space and time using numerical calculation methods (such as finite element analysis, finite difference or finite volume methods, etc.). Such a calculation method can be iteratively performed using a time stepping scheme that allows obtaining the evolution of the model state, which is updated at a sufficiently high rhythm to allow the changes in the internal anatomical structure to be captured and recorded at the desired resolution. Different sub-regions of the anatomical region 202 can be modeled with different mathematical models. In particular, different sub-regions can be modeled by a set of different equations that take into account the different tissue types present in the biomechanical model 300.

[0217] For example, establishing the biomechanical model 300 may include dividing the anatomical region 202 into at least a first anatomical sub-region 202a exhibiting a first set of physical properties and a second anatomical sub-region 202b exhibiting a second set of physical properties different from the first set of physical properties (see Figure 1). For the first anatomical sub-region 202a, the grid-based model 300 can be assigned a first grid-based sub-model applicable to the first set of physical properties; and for the second anatomical sub-region 202b, the grid-based model 300 can be assigned a second grid-based sub-model applicable to the second set of physical properties.

[0218] The first anatomical sub-region 202a and the second anatomical sub-region 202b may share a common boundary, and the method of building the biomechanical model 300 may include the step of assigning a common boundary condition to the first and second point-based or mesh-based sub-models in the common boundary.

[0219] There are many mathematical models that describe the behavior of materials. These models include linear elastic models, where linear elasticity describes the deformation of a body under external or internal loads as being linearly proportional to the load. In these models, Hooke's law represents the behavior of the material and relates the unknown stresses and strains. Known mathematical models also include hyperelastic models, where the material not only displays nonlinear material behavior, but also large deformations and stress-strain relationships are derived from the strain energy density function. Examples of hyperelastic material models include the Neo-Hookean model and the Mooney-Rivlin model.

[0220] The biomechanical model 300 may incorporate any of these mathematical models.

[0221] In general, the biomechanical model 300 may be defined by a system of partial differential equations F with source terms f, boundary conditions g and h, and model parameters p:

[0222]

[0223] The model state variable u is a displacement field representing the space-time displacement of the domain Ω, and t represents the time dimension.

[0224] Providing the radiation therapy system 100 also involves training a model on the cyclic or semi-cyclic motion that the radiation therapy system 100 is configured to predict and compensate for (ie, respiratory motion in the present embodiment).

[0225] Preferably, the training involves representing the model 300 as a graph in a graph network system and applying a loss function to the inferred future states provided by the graph network system.

[0226] Representing the model 300 as a graph in a graph network system involves establishing graph nodes, graph edges, and graph global elements having characteristics that reflect the static and dynamic variables of the biomechanical model 300 .

[0227] Graph networks are known, for example, from the aforementioned papers “Learning to Simulate Complex Physics with Graph Networks,” Sanchez-Gonzalez et al., arXiv:2002.09405v2, and “Learning Grid-Based Simulations with Graph Networks,” Pfaff et al., arXiv:2010.03409v4, and a graph network according to the radiation therapy system 100 may advantageously be implemented using the teachings of either of these papers as a starting point.

[0228] Figure 5 Schematically shows the representation Figure 4 400 of the biomechanical model 300 shown in FIG. 4 , the graph includes graph nodes 402 and graph edges 404. In the present embodiment, the graph nodes 402 and the graph edges 404 may be designated to represent the mesh nodes 302 and the mesh vertices 304, respectively, and the state variables representing the spatiotemporal displacement of the anatomical region 202 (e.g., represented by the displacement field u) may be embedded in the graph edges 404. In addition, the internal forces within the biomechanical model 300 may be encoded as directed graph edge features between two adjacent graph nodes, representing the force applied by one mesh node to a neighboring mesh node.

[0229] Figure 6 The main building blocks of a graph network system or framework 500 implemented within the context of the present embodiment of the radiation therapy system 100 are schematically illustrated.

[0230] The graph network system 500 includes an input 502, a graph processing neural network 504, an updater 506, and an output 508. The graph processing neural network 504 includes an encoder 510, a processor 512, and a decoder 514.

[0231] The encoder 510 is configured to receive the current state M of the mesh-based biomechanics model 300 from the input 502. t , and the current state M from the biomechanical model 300 t The input graph G 0 to encode.

[0232] The processor 512 is configured to process the input graph G 0 and provides the output graph G with updated node and edge embeddings M Processor 512 includes one or more graph network blocks 516 (see Figure 7), which takes a graph as input, performs computation on the graph structure, and returns a graph as output. Each graph network block contains three sub-functions - one for each of the graph edges, graph nodes, and graph global elements. The sub-functions can be implemented by potentially using any class of neural networks. Each of the sub-functions can be implemented with, for example, a multi-layer perceptron (MLP), a recurrent neural network (RNN), a long short-term memory (LSTM) recurrent neural network, or a convolutional neural network (CNN).

[0233] The implementation of the subfunction can be based on DeepMind's Sonnet library (https: / / github.com / deepmind / sonnet).

[0234] The decoder 514 is configured to decode the output graph G M , extracting output features corresponding to static and dynamic variables of the biomechanical model 300.

[0235] The updater 506 is configured to update the output features to generate an inferred future state M of the mesh-based biomechanical model 300. t+1 , and the inferred state M t+1 Provided to output 508.

[0236] Training the biomechanical model 300 preferably involves applying a loss function to the inferred state provided by the updater and to a real-world dataset on an anatomical region corresponding to the modeled anatomical region 202. The real-world dataset may be based on a 4D computed tomography (4D-CT) scan of an anatomical region corresponding to the modeled anatomical region 202. To this end, Hounsfield units, which impart opacity to tissue material to X-rays, should preferably be included in the model 300 as a variable.

[0237] Alternatively or additionally, the training may be based on data sets acquired from 4D Magnetic Resonance Imaging (4D-MRI) or Ultrasound Imaging.

[0238] For training the breathing motion, the dataset may preferably be based on the evolution of images covering the modeled anatomical region 202 during the breathing cycle. For training the model, at least two image datasets of the same patient may be used, each capturing a different breathing pattern. However, preferably, images from a larger patient population may be used, which exhibit different breathing patterns (inter-patient and intra-patient variations).

[0239] Now return to reference Figure 1 and Figure 2 , the radiotherapy system 100 comprises at least one beam source 102 arranged to provide an ionizing radiation beam 104. The radiation beam 104 may be a particle beam, such as a proton beam, or a photon beam, such as an X-ray beam.

[0240] The radiation therapy system 100 further includes a beam supervision system 106 configured to control the position and alignment in space of each beam source 102. The control may preferably include controlling three variables defining the position of each particle beam source, e.g., represented by Cartesian coordinates x, y, z, and / or variables defining the pitch, yaw and / or roll of the beam source, e.g., represented by rotation angles measured about orthogonal pitch, yaw and / or roll axes, e.g., Figure 1 and Figure 2 Schematically shown in FIG.

[0241] The radiation therapy system 100 also includes a control system 110 configured to control the beam supervision system 106 to track the irradiation target 204. To this end, the control system 110 includes a computer-implemented graph network system or framework 500 in which the above-described trained biomechanical model 300 is implemented. The graph network system 500 can advantageously be the same type of graph network system 500 as the graph network system used to train the model 300.

[0242] The radiation therapy system 100 also includes a monitoring system 120 arranged to monitor the anatomical region 202 to receive updated data regarding at least one aspect of the current state of the anatomical region 202. "Updated data" herein refers to data that is sufficiently recent to provide information regarding the current state of the anatomical region 202. Thus, while the updated data should preferably be real-time or near real-time data, the most recent data may correspond to updated data having a cyclic or near-cyclic motion of a slowly evolving state.

[0243] The monitoring system 120 can, for example, be configured to monitor the position and / or orientation in space of a boundary region 206 of an anatomical region 202, such as the envelope of a chest of a radiotherapy patient, i.e., the outer contour of the chest (body surface), for example, indicated by skin markers. However, the monitored data may additionally or alternatively include the position and / or orientation in space of an internal structure of the anatomical region, for example, the position and / or orientation in space of one or more fiducial markers 208. The data may be received by the monitoring system 120 from an imaging system 122 (e.g., a camera operating within the visible electromagnetic spectrum) or an ultrasound imaging system. Alternatively or additionally, a 2D image (e.g., an MRI or X-ray projection image) may be acquired by tracking a landmark in the patient (whether it is a fiducial or other anatomical landmark). However, a 3D-MRI or 3D-CT system may be less suitable for delivering updated data because such a system is typically associated with relatively long acquisition and reconstruction times (e.g., high data latency). However, for slowly evolving cyclic or semi-cyclic motions, 3D-MRI and 3D-CT systems may alternatively or additionally be used to provide updated data.

[0244] Preferably, the monitoring system 120 is also arranged to monitor the beam sources 102 to receive updated data regarding the current position and / or alignment of at least one beam source 102. This data may be received by the monitoring system 120 from the beam supervision system 130 with low latency.

[0245] As previously described, the biomechanical model 300 of the anatomical region 202 includes mesh nodes 302 and mesh vertices 304, wherein each mesh node 302 is associated with a world space coordinate x of the mesh node 302 and a state variable u representing a spatiotemporal displacement of the anatomical region 202, which state variable u may be a displacement field.

[0246] Therefore, in the graph network system 500 implemented in the control system 110, the trained biomechanical model 300 can be represented as a graph 400, where the graph nodes 402 and the graph edges 404 are designated to represent the mesh nodes 302 and the mesh vertices 304, respectively, and the state variables represented by the displacement field u can be embedded in the graph edges. In addition, as previously described, the internal forces within the biomechanical model 300 can be encoded as directed graph edge features between two adjacent graph nodes, representing the force applied by one mesh node to a neighboring mesh node.

[0247] Thus, the nodes, edges, and global elements of graph 400 may have the following characteristics:

[0248]

[0249] The dynamics of the system, ie, the state variables u and their derivatives are embedded in the graph edges. In processor 512, each graph network block 516 takes an input graph and transforms the dynamics defined on the edges to produce an output graph.

[0250] The features received by each graph edge determine the physical interactions and properties from the biomechanical model encoded in the network.

[0251] The graph edges embed the relative displacement field u and its norm |u|. In addition, the internal force f in the model ij It is encoded as a directed edge feature between two adjacent nodes, representing the force exerted by one node on its neighbor node.

[0252] The differential operator can be approximated by message passing between adjacent graph nodes, which act as sub-functions (f M ) are embedded in the graph edges. These functions take as input edge features (e.g. derivatives and forces) and sender and receiver node features (coordinates, material properties, etc.) and output higher-order derivatives of the nodes. These in turn are used to integrate the dynamic variable x.

[0253] Graph nodes can also embed functions (fV ), independent of messages passed between graph nodes, e.g., transformations of material properties.

[0254]

[0255] Here, e ij M and e ij W The mesh and the world edge respectively.

[0256] The graph processing neural network 504 processes graph node features, graph edge features, and global features, and may output the following:

[0257]

[0258] In updater 506, the dynamic quantity x at time t+1 is then obtained by integrating the derivative at time t. i For the first-order model, the dynamic quantity x is updated using the first-order derivative output by the graph processing neural network 504 i , and for the second-order model, x' i Embed as node features and use x" i The dynamic quantity x is calculated according to the following formula i To perform integration:

[0259] x i t+1 =x' i +x i t (for first-order models)

[0260] or

[0261] x i t+1 =x” i +2x i t -x i t-1 (for second-order models)

[0262] Thus, at output 508, a graph having inferred features x is provided. i t+1 The inferred future state M of the model t+1 , that is, the predicted future position of the grid node 302 of the model 300 .

[0263] During operation of the radiation therapy system 100, the current state M of the training model 300 is established. t Preferably, the current state M is established t Involves changing the current state M tAssociated with data (i.e., real-world data) regarding the current position and / or orientation in space of the anatomical region 202. The data may preferably be provided by the monitoring system 120 and may, for example, be based on updated data regarding the position and / or orientation in space of a boundary region of the anatomical region 202, such as an envelope of a radiotherapy patient's chest, i.e., the outer contour of the chest (body surface), and / or the position and / or orientation in space of internal structures of the anatomical region, such as the position and / or orientation in space of one or more fiducial markers.

[0264] If the current state M t The current state M of the model 300 may be adjusted or updated if it is not related to the real world data provided by the monitoring system 120. t , to better represent real-world data. If the current state M t If the correlation with real-world data is poor, for example, below a predetermined limit, the radiation therapy session may be aborted.

[0265] If the current state M is found t represents real-world data, the graph network system 500 provides the inferred future state M of the training model t+1 , and the inferred future state M based on the training model 300 t+1 To control the position and / or alignment of at least one beam source 102 to track the irradiation target 204.

[0266] The radiation therapy system 100 may be configured to intermittently provide the inferred future state M of the model 300. t+1 , with a period in the range of 20ms to 100ms, for example, every 50ms.

[0267] According to the above embodiment, whenever the current state M of the training model 300 is established, t When the current state M t is associated with real-world data. In this case, the monitoring system update rate should not be lower than the model update rate, for example, every 50ms. According to an alternative embodiment, the current state M t It can be associated with real-world data less frequently, for example, every second or third time the current state M is established t In such an embodiment, the current state M t The state M can be mainly based on the previous inference of the model t-1 、M t-2 、M t-3 At least one of the above.

[0268] The radiotherapy system 100 according to the present disclosure allows prediction of the future position, alignment, and / or deformation of the irradiation target 204, thereby giving the beam management system 106 time to react to changes in the state of the irradiation target 204. In addition, the radiotherapy system 100 allows accurate tracking of the irradiation target 204 even though updated real-world data regarding the position, alignment, and / or deformation of the irradiation target 204 is not available.

[0269] Furthermore, since the anatomical region 202 is modeled to include organs and tissues surrounding the irradiation target 204, the movement and deformation of the risk organs can also be predicted, thereby allowing the control system 110 to instruct the beam supervision system 106 to position and align the beam source 102 so that irradiation of such organs can be avoided or at least minimized. Additionally or alternatively, if the inferred future state indicates that the radiation beam is at risk of intersecting the risk organ, the control system 110 can be configured to instruct the beam supervision system 106 to block the radiation beam or turn off the corresponding beam source. Blocking the radiation beam can be achieved by applying a gating function to the beam source.

[0270] Additionally or alternatively, the control system 110 may be configured to instruct the beam supervision system 106 to block the radiation beam or shut down the corresponding beam source if the inferred future state indicates that the irradiation target will venture outside the trajectory of the ionizing radiation beam, for example because the maximum range of the alignment system of the beam source may not be able to accommodate the predicted movement of the irradiation target.

[0271] As previously described, the training model 300 may advantageously include variables characterizing the planned and / or maximum allowed radiation dose in or at each grid node 302 or voxel, and variables characterizing the cumulative radiation dose in or at each grid node 302. By receiving updated information about the position and alignment of the beam source, and therefore about the locations where radiation energy is deposited in the training model 300, the control system 110 may determine the optimal radiation dose for each inferred future state M of the training model. t+1 The accumulated radiation dose in or at each grid node 302 is updated, allowing the control system 110 to instruct the beam supervision system 106 to position and aim the beam source 102 so that the irradiation target 204 receives the planned radiation dose without exceeding the maximum allowed radiation dose to other portions of the anatomical region 202 .

[0272] Figure 8 An embodiment of a method of establishing a biomechanical model of an anatomical region of a radiation therapy patient for use in a radiation therapy system (eg, for use in the radiation therapy system 100 described above) is shown.

[0273] The method comprises a step of establishing a point-based or mesh-based model of the modeled anatomical region 802. The point-based or mesh-based model (300) may be, for example, the mesh-based model 300 discussed above.

[0274] The method further includes a step 804 of associating static and / or dynamic features with points or grid nodes of the point-based or grid-based model. The points or grid nodes of the point-based or grid-based model may be, for example, the above-mentioned grid nodes 302 of the grid-based model 300, and the static and / or dynamic features include world space coordinates of the points or grid nodes. For example, the world space coordinates of the points or grid nodes may be the above-mentioned world space coordinates x of the grid node 302. The static and / or dynamic features also include a state variable representing the spatiotemporal displacement of the anatomical region. For example, the state variable representing the spatiotemporal displacement of the anatomical region may be the above-mentioned state variable u representing the spatiotemporal displacement of the anatomical region 202.

[0275] For each point or grid node, the static and / or dynamic features also include at least one feature characterizing body tissue in the anatomical region. The at least one feature characterizing body tissue in the anatomical region may include at least one variable characterizing the interaction between the anatomical region and the ionizing radiation beam, such as at least one of: linear stopping power; relative stopping power; radiodensity and radiopacity.

[0276] The method further includes a step 806 of representing the model as a graph in the graph neural network system. For example, step 806 may include representing the above model 300 as a graph in the graph neural network system 500.

[0277] The method also includes a step of training the model on cyclic or semi-cyclic motion of the anatomical region 808. For example, step 808 may include training the model 300 on cyclic or semi-cyclic motion of the anatomical region 202, as previously described.

[0278] Step 806 of representing the model as a graph in a graph neural network system may include establishing graph nodes, graph edges, and graph global elements having characteristics that reflect static and dynamic variables of the biomechanical model (e.g., the characteristics of the model 300 described above).

[0279] For example, when the above-mentioned model 300 is represented as a graph in the above-mentioned graph neural network system 500, the graph network system 500 can be configured to include an input 502, a graph processing neural network 504, an updater 506, and an output 508, as described above, and the graph processing neural network 504 can include an encoder 510, a processor 512, and a decoder 514. The encoder 510 can be configured to receive the current state M of the grid-based biomechanics model 300 from the input 502. t , and the current state M from the biomechanical model 300 t The input graph G 0 The processor 512 may be configured to process the input graph G 0and provides the output graph G with updated node and edge embeddings M The decoder 514 may be configured to decode the output graph G M , extracting output features corresponding to the static and dynamic variables of the biomechanical model 300, and the updater 506 can be configured to update the output features to produce an inferred future state M of the mesh-based biomechanical model 300 t+1 , and the inferred state M t+1 Provided to output 508.

[0280] Step 808 of training the model on the cyclic or semi-cyclic motion of the anatomical region may include applying a loss function to the inferred state M provided by the updater 506 t+1 and a real-world dataset applied to an anatomical region corresponding to the modeled anatomical region. The real-world dataset may include at least one of a 4D computed tomography (4D-CT) scan, a 4D magnetic resonance imaging (4D-MRI), and ultrasound imaging. The real-world dataset may include an evolution of an image covering the modeled anatomical region during a respiratory cycle.

[0281] The point-based or grid-based model may incorporate at least one of a linear elastic model and a hyperelastic model adapted to the physical properties of the anatomical region.

[0282] The step 802 of establishing a point-based or grid-based model of the modeled anatomical region may include: a first sub-step 802a of dividing the anatomical region into at least a first anatomical sub-region displaying a first set of physical properties and a second anatomical sub-region displaying a second set of physical properties different from the first set of physical properties; a second sub-step 802b of assigning, for the first anatomical sub-region, a first point-based or grid-based sub-model applicable to the first set of physical properties to the point-based or grid-based model; and a third sub-step 802c of assigning, for the second anatomical sub-region, a second point-based or grid-based sub-model applicable to the second set of physical properties to the point-based or grid-based model. In the case where the first and second anatomical sub-regions share a common boundary, sub-steps 802b and 802c may include assigning a common boundary condition to the first and second point-based or grid-based sub-models in the common boundary.

[0283] Fig. 9 An embodiment of a computer-implemented method for planning at least a portion of a radiation therapy procedure to be performed on an anatomical region of a radiation therapy patient is shown in FIG. Figure 8The method discussed has a first step 902 of establishing a biomechanical model of the anatomical region, and a second step 904 of assigning at least one of a planned dose and a maximum allowed dose to each point or grid node, thereby allowing the model to compare absorbed doses at different points or grid nodes with the planned or maximum allowed doses during subsequent radiotherapy sessions. As previously described, the radiotherapy system in which the model is implemented can then control dose delivery and beam alignment based on the difference between the planned dose and the absorbed dose, so that the irradiated target receives the planned radiation dose without exceeding the maximum allowed radiation dose of other parts of the anatomical region.

[0284] In the foregoing description, various aspects of the device according to the present invention have been described with reference to illustrative embodiments. For the purpose of explanation, specific numbers, systems and configurations are listed to provide a thorough understanding of the device and its working principle. However, this description is not intended to be interpreted as limiting. Various modifications and changes to the illustrative embodiments and other embodiments of the device that are obvious to those skilled in the art of the disclosed subject matter are considered to fall within the scope of the present invention defined by the appended claims.

Claims

1. A method of establishing a biomechanical model (300) of an anatomical region (202) of a patient (200) for radiotherapy in a radiotherapy system (100), the method comprising the following steps: (a) establishing a point-based or grid-based model (300) of the anatomical region (202); (b) associating static and / or dynamic features with points or mesh nodes (302) of the point-based or mesh-based model (300), the features comprising world space coordinates (x) of the point or mesh node (302) and state variables (u) representing the spatiotemporal displacement of the anatomical region (202), and, for each point or mesh node (302), at least one feature characterizing body tissue in the anatomical region; (c) representing the model (300) as a graph in a graph neural network system (500); and (d) training the model (300) on cyclic or semi-cyclic motion of the anatomical region (202).

2. The method according to claim 1, wherein: The at least one characteristic characterizing body tissue in the anatomical region includes at least one variable characterizing an interaction between the anatomical region (202) and a beam of ionizing radiation.

3. The method according to claim 2, wherein: The at least one characteristic characterizing an interaction between the anatomical region (202) and the beam of ionizing radiation includes at least one of: linear stopping power; Relative stopping power; radiodensity and radiopacity.

4. The method according to any one of claims 1 to 3, wherein: The step of representing the model (300) as a graph in a graph neural network system (500) includes: establishing graph nodes, graph edges, and graph global elements having characteristics reflecting static and dynamic variables of the biomechanical model (300); wherein the graph network system (500) comprises an input (502), a graph processing neural network (504), an updater (506), and an output (508), and wherein the graph processing neural network (504) comprises an encoder (510), a processor (512), and a decoder (514); The encoder (510) is configured to receive a current state (M) of the mesh-based biomechanical model (300) from the input (502). t ), and the current state (M) from the biomechanical model (300) t )’s input graph (G 0 ) for encoding; The processor (512) is configured to process the input graph (G 0 ) and provides an output graph (G M ); The decoder (514) is configured to decode the output graph (G M ), extracting output features corresponding to the static and dynamic variables of the biomechanical model (300); and wherein the updater (506) is configured to update the output features to generate an inferred future state (M) of the mesh-based biomechanical model (300) t+1 ), and the inferred state (M t+1 ) is provided to the output (508).

5. The method according to claim 4, wherein: The step of training the model (300) on the cyclic or semi-cyclic motion of the anatomical region (202) comprises applying a loss function to the inferred state (M) provided by the updater (506) t+1 ) and a real-world dataset applied to an anatomical region corresponding to the modeled anatomical region (202).

6. The method according to claim 5, wherein: The real-world dataset comprises at least one of: 4D computed tomography 4D-CT scan, 4D magnetic resonance imaging 4D-MRI and ultrasound imaging.

7. The method according to any one of claims 5 and 6, wherein: The real-world dataset includes an evolution of an image covering the modeled anatomical region (202) during a respiratory cycle.

8. A method according to any one of the preceding claims, wherein: The point-based or grid-based model incorporates at least one of: a linear elastic model and a hyperelastic model adapted to the physical properties of the anatomical region (202).

9. The method according to any one of the preceding claims, comprising the steps of: - dividing the anatomical region (202) into at least a first anatomical sub-region (202a) exhibiting a first set of physical properties and a second anatomical sub-region (202b) exhibiting a second set of physical properties different from the first set of physical properties; - for said first anatomical sub-region (202a), assigning to said point-based or grid-based model (300) a first point-based or grid-based sub-model suitable for said first set of physical properties; as well as - for said second anatomical sub-region (202b), assigning to said point-based or grid-based model (300) a second point-based or grid-based sub-model adapted for said second set of physical properties.

10. The method according to claim 9, wherein: The first and second anatomical sub-regions share a common boundary, and the method comprises the step of assigning a common boundary condition to the first and second point-based or mesh-based sub-models in the common boundary.

11. A computer-implemented method of planning at least a portion of a radiation therapy procedure to be performed on an anatomical region (202) of a radiation therapy patient (200), comprising: - establishing a biomechanical model (300) of the anatomical region (202) according to any one of claims 1 to 10; and - Each point or grid node is assigned at least one of: a planned dose and a maximum allowed dose.

12. A radiotherapy system (100), comprising: - at least one beam source (102) arranged to provide a beam of ionizing radiation (104) to an irradiation target (204) located in the anatomical region (202); - a beam supervision system (106) configured to control the position and / or alignment of the at least one beam source (102); and - a monitoring system (120) arranged to monitor the anatomical region (202), Features: - the control system (110) comprises a computer-implemented graph network system (500) in which a model (300) of the anatomical region (202) trained on cyclic or semi-cyclic motions of the anatomical region (202) is implemented; The graph network system (500) is configured to establish a current state (M) of the training model (300) based on data received from the monitoring system (120) regarding a current position and / or a current orientation in space of the anatomical region (202). t ), and is configured to be based on the current state (M t ) to establish the inferred future state (M) of the training model (300) t+1 );as well as The control system (110) is configured to instruct the beam supervision system (106) to determine the future state based on the inferred future state (M t +1 ) to control the ionizing radiation beam (104) of the at least one beam source (102).

13. The radiation therapy system (100) of claim 12, wherein: The control system (110) is configured to instruct the beam supervision system (106) to determine the future state based on the inferred future state (M t+1 ) to control the ionizing radiation beam (104) of the at least one beam source (102) so that the at least one beam source (102) tracks the irradiation target (204).

14. The radiation therapy system (100) according to any one of claims 12 and 13, wherein: The control system (110) is configured to: if the inferred future state (M t+1 ) indicates that the irradiation target will venture outside the trajectory of the ionizing radiation beam (104), the control system (110) instructs the beam supervision system (106) to perform a t+1 ) to control the ionizing radiation beam (104) of the at least one beam source (102) to block the ionizing radiation beam (104) of the at least one beam source (102) or shut down the at least one beam source (102).

15. The radiation therapy system (100) according to any one of claims 12 to 14, wherein: The control system (110) is configured to: if the inferred future state (M t+1 ) indicates that the trajectory of the ionizing radiation beam (104) is at risk of intersecting the organ at risk, the control system (110) instructs the beam supervision system (106) to perform a predetermined operation based on the inferred future state (M t+1 ) to control the ionizing radiation beam (104) of the at least one beam source (102) to block the ionizing radiation beam (104) of the at least one beam source (102) or shut down the at least one beam source (102).

16. The radiation therapy system (100) according to any one of claims 12 to 15, wherein: The training model (300) is a point-based or grid-based model (300), comprising static and / or dynamic features associated with a point or grid node (302) of the point-based or grid-based model (300), the features comprising a world space coordinate (x) of the point or grid node (302) and a state variable (u) representing a spatiotemporal displacement of the anatomical region (202).

17. The radiation therapy system (100) according to any one of claims 12 to 16, wherein: The graph network system (500) is configured to represent the training model (300) as a graph (400) including graph nodes (402), graph edges (404), and graph global elements having characteristics reflecting static and dynamic variables of the training model (300).

18. The radiation therapy system (100) according to any one of claims 12 to 17, wherein: The graph network system (500) is configured to convert the established current state (M) of the training model (300) into t ) is related to the data received from the monitoring system (120).

19. The radiation therapy system (100) of claim 18, wherein: The monitoring system (120) is configured to monitor at least one of: the position and / or orientation in space of a boundary region (206) of the anatomical region (202); and the position and / or orientation in space of a reference marker (208) in the anatomical region (202).

20. The radiation therapy system (100) according to any one of claims 12 to 19, in, The variables include at least one variable characterizing an interaction between the anatomical region (202) and the ionizing radiation beam, and at least one variable characterizing at least one of: a planned radiation dose; a maximum allowed radiation dose; and a cumulative radiation dose, Wherein, for each inferred future state (M t+1 ), the graph network system (500) is configured to update the variable representing the cumulative radiation dose, and Wherein, the control system (110) is configured to instruct the beam supervision system (106) to control the position and / or alignment of the at least one beam source (102) so that the irradiation target (204) receives a planned radiation dose without exceeding a maximum allowable radiation dose to other parts of the anatomical region (202).

21. The radiation therapy system (100) according to any one of claims 12 to 20, wherein: The monitoring system (120) is arranged to monitor the beam supervision system (106).

22. The radiation therapy system (100) according to any one of claims 12 to 21, wherein: The training model (300) of the anatomical region (202) has been established according to any one of claims 1-10.

Images