Positioning system for radiotherapy
By using optimization controllers and neural networks in the radiotherapy positioning system for patient positioning and dose distribution optimization, the problem of high computing resources in the prior art is solved, and faster and more efficient patient positioning and treatment plan optimization is achieved, improving the improvement of the improvement of the patient positioning and treatment plan. The accuracy and efficiency of treatment.
Patent Information
- Application Number
- CN202380022941.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-02-22
- Filing Date
- 2023-02-21
- Publication Date
- 2024-11-08
AI Technical Summary
The existing radiotherapy positioning system consumes a lot of resources and time computationally, making it difficult to effectively deal with changes in patient location and tumor shape during treatment.
Using a positioning system including an optimization controller and a neural network-based control module, patients are accurately positioned and dose distribution optimization through neuromorphic data processing and convolutional neural networks, combining geometric rigid registration and dose guidance to reduce computational burden.
Faster and efficient patient positioning is achieved, reducing computing resource requirements, improving the accuracy and efficiency of treatment plans, and reducing exposure to healthy tissues.
Smart Images

Figure CN118922229A_ABST
Abstract
Description
[0001] field
[0002] The present invention relates to positioning systems for use in radiation therapy.
[0003] The invention also relates to a training system.
[0004] The invention also relates to a training method.
[0005] background
[0006] Radiotherapy (also called radiation therapy) is a cancer treatment that uses high doses of ionizing radiation to kill cancer cells and shrink tumors. In preparation for treatment, radiation oncologists work with medical physicists and radiation therapists to design a treatment plan that aims to result in effective irradiation of malignant areas while minimizing exposure to healthy tissue.
[0007] The radiation oncologist bases the treatment plan on the location of the malignant area obtained from the 3D / 4D CT images of the patient before the treatment. In this regard, it should be noted that throughout the specification, the wording "image" is used to refer to a volumetric image data set or scan obtained from a volumetric imaging method. Non-limiting examples of volumetric imaging methods are CT, CBCT, PET, MRI and synCT. During treatment, which usually includes multiple treatment phases, the actual situation may be different from what is observed in the 3D / 4D images. This may be due to various reasons, such as the position of the patient, changes in the shape of the tumor, loss or gain of the patient's weight, and internal movements caused by breathing and the patient's heartbeat.
[0008] Haehnle et al., 2017 Phys. Med. Biol. 62 165 describe a method for patient positioning for interactive multi-target dose guidance. The method proposed therein aims to reposition the patient so as to obtain the best dose distribution, that is, the dose distribution that best matches the effective irradiation target of the malignant area, while not harming the healthy area, in particular, not harming the organ at risk (OAR). The method proposed therein comprises a first step, in which a 3D cone beam computed tomography (CBCT) scan is obtained when the patient is in the treatment position. In the first step, a rigid alignment is calculated, for which the anatomical structure of the patient in the original image best matches the anatomical structure in the CBCT scan. In the second step, the dose distribution is calculated, and in the case of this rigid realignment and for multiple alignments different from this alignment in the space C that can be offset, the original treatment plan is applied to the patient to achieve the dose distribution. In addition, the dose distribution of the intermediate alignment is calculated by interpolation. The disadvantage of the known method is that the dose calculation for various alignments imposes a large amount of computational burden and involves a large amount of computational time.
[0009] Overview
[0010] It is an object of the present invention to provide an improved positioning system requiring fewer computing resources.
[0011] Hereby, an improved positioning system as defined in claim 1 is provided.
[0012] The improved system claimed therein positions a patient for treatment in a radiation beam based on patient-specific data including a first image of the patient. The first image is typically a 3D / 4D image, obtained by a volume imaging method as mentioned above, such as by CT, or less frequently by an MRI or PET / CT scanner. The patient-specific data is provided by a medical expert (e.g., an oncologist), for example, using a planning tool. During the treatment planning process, the targets and organs at risk are segmented, a sufficient number of beam directions or arc lengths are established relative to the isocenter, and based on the dose to the tumor and the target to the organs at risk prescribed by the physician, the plan is optimized and calculated to determine the optimal dose distribution for the specific patient. Treatment planning is typically performed over a few weeks of treatment, 5 days a week.
[0013] The improved system further comprises an imaging device configured to provide a second image (scan) of the patient, which includes the body part to be treated. Furthermore, the imaging device for providing the second image is typically a 3D imaging device, such as a CT scanner or an MRI scanner. The second image provides updated information about the location and size of tissues and structures in the patient's body, preferably immediately prior to the treatment phase.
[0014] The improved system further includes a positioning device (e.g., a treatment couch) for maintaining a patient in a variable position and / or orientation in the radiation beam for precise intervention on a specified portion of the patient. The positioning device is controlled by an optimization controller included in the improved system. The optimization controller is configured to provide registration control data, i.e., control data for controlling the position and / or orientation of the positioning device to guide the positioning device based on the first image and the second image. The optimization controller includes a dose-based control module, which is configured to calculate an optimized repositioning of the patient by performing a dose-guided optimization. To this end, the dose-based control module is configured to perform at least the following operations.
[0015] It performs a neuromorphic data processing operation on first image data obtained from a first image to present first feature map data. It also performs the same neuromorphic data processing operation on second image data obtained from a second image to present second feature map data. Subsequently, it concatenates the first feature map data and the second feature map data to provide concatenated feature map data. It then performs a further neuromorphic data processing operation on the concatenated feature map data to provide a correction vector indicating the desired registration of the patient for dose-based optimization. The concatenated feature map data is, for example, a feature map comprising the first feature map data and the second feature map data. Due to the fact that the steps of the method are performed in this order, i.e., first the corresponding feature map data are extracted from each image, and then the feature map data thus obtained are concatenated, the same neural network structure can be trained and used with different types of images (e.g., CBCT / MRI). Therefore, registration between different image types (CT+CBCT or CT+MRI) can be performed by the same neural network.
[0016] As described in more detail below, the dose-based control module specified for the improved system can be easily trained to perform registration in an efficient manner.
[0017] In an embodiment, the improved system further includes a control module and a transformation module based on geometry. The control module based on geometry is configured to perform rigid registration of geometry to provide a first estimate for the required registration of the patient based on the first image and the second image. This rigid registration of geometry is based on the registration of rigid structures in the patient's body (e.g., bones), or the registration of markers intentionally introduced when planning treatment. This rigid registration of geometry is sufficient under the assumption that no morphological changes occur in the patient's body. However, in practice, such morphological changes do occur, for example, due to changes in body weight and changes in the size of the tumor to be treated. Therefore, the control module based on geometry is provided as a supplement to the control module based on dose rather than as a replacement. However, the registration provided by the control module based on geometry is a first approximation of the starting point used as a control module based on dose. In order to compensate for the correction provided by the first estimate, a transformation module is provided to transform the first image or the second image to compensate for the registration difference indicated by the first estimate. The control module based on dose in this embodiment of the improved system is configured to determine an estimate of the required additional registration of the patient based on the first image and the second image, wherein one of the first image and the second image is transformed. For example, the transformation module transforms the second image according to the first approximation as if the patient's registration had been corrected according to the first approximation, and the dose-based control module determines an estimate of the additional registration correction required based on the first image and the transformed second image. Optionally, the transformation module transforms the first image according to the inverse of the transformation specified by the first approximation, and the dose-based control module determines an estimate of the additional registration correction required based on the inverse transformed first image and the second image. An advantage of an embodiment of the improved system further comprising a geometry-based control module and a transformation module as described above is that the dose-based control module can be more easily trained. The geometry-based control module and the transformation module can be implemented in a straightforward manner requiring only a modest computational effort.
[0018] In an embodiment of the improved system, the dose-based control module includes mutually identical first and second convolutional neuromorphic processing branches, a neuromorphic connection stage, and a fully connected neuromorphic stage.
[0019] In these embodiments, the first convolutional neuromorphic processing branch is configured to receive first image data obtained from a first image and provide first feature map data based on the received first image data.
[0020] The second convolutional neuromorphic processing branch is configured to receive second image data obtained from the second image and provide second feature map data based on the received second image data. It should be noted that, conversely, a single convolutional neuromorphic processing unit can be used in a timeshared manner to calculate the first feature map data and the second feature map data from the first and second image data, respectively.
[0021] The neuromorphic concatenation stage is configured to receive the first feature map data and the second feature map data and provide concatenated feature map data based on the first feature map data and the second feature map data.
[0022] The fully connected neuromorphic stage is configured to receive the concatenated feature map data and provide a correction vector indicating a correction in the registration.
[0023] In an exemplary embodiment, mutually identical convolutional neuromorphic processing branches each include multiple stages with convolutional neuromorphic layers. Typically, mutually identical convolutional neuromorphic processing branches or a single convolutional neuromorphic processing branch applied in a time-sharing manner includes up to ten stages, such as 5 stages.
[0024] In addition to the convolutional neuromorphic layer, the stage may also include one or more of a batch normalization layer, an activation layer (e.g., a ReLU layer (e.g., a leaky ReLU activation layer)), a pooling layer (e.g., a max pooling layer, etc.).
[0025] An embodiment of the improved system further comprises a third control module configured to calculate a correction vector for dose guidance optimization by applying a gradient descent algorithm. Also in this embodiment, the dose-based control module is optionally combined with a geometry-based control module.
[0026] In this example of the embodiment, the third control module includes the following components.
[0027] a) A transformation vector modifier configured to generate, for each transformation vector at its input, a set of modified transformation vectors.
[0028] b) An image transformation module configured to provide a set of modified second images by transforming the second image according to respective modified transformation vectors of the set of modified transformation vectors.
[0029] c) A dose simulation module configured to provide an estimated spatial dose distribution for each modified second image.
[0030] d) A dose volume histogram calculation module, which is used to calculate the expected dose volume histogram generated by each estimated spatial dose distribution.
[0031] e) An evaluation module for calculating a quality measure indicative of an expected quality of the treatment based on the calculated expected dose-volume histogram for each estimated spatial dose distribution.
[0032] f) A gradient descent module for computing a single next iteration transformation vector based on respective values of expected quality determined for the modified transformation vectors in the set of modified transformation vectors.
[0033] In operation, the third control module iteratively calculates the transformation vector until further iterations do not result in a significant increase in the quality measure. Optionally, a predetermined maximum number of iterations may be specified. As a further option, the iteration stops once one of these two conditions is met.
[0034] The simulation module in this example can estimate the spatial dose distribution using Monte Carlo simulation.
[0035] The quality measure can also be based on an estimate of the normal tissue complication probability (NTCP), which indicates the probability that normal tissue is affected by the treatment. In an embodiment, the third module further includes a treatment effect calculation module, which provides an indication of this probability associated with the spatial dose distribution estimated by the dose simulation module. In this calculation, the treatment effect calculation module can consider one or more RT structural information and information about prognostic factors. In this embodiment, the evaluation module is configured to calculate the quality measure based on the calculated expected dose volume histogram and based on the estimate of the normal tissue complication probability.
[0036] According to a second aspect of the present disclosure, there is provided a training system for training a dosage-based control module, comprising:
[0037] a) a first input for receiving a first image of a patient, the first image comprising tissue label data and dose specification data;
[0038] b) a second input for receiving a second image of the patient, the second image comprising a designated portion of the patient to be treated and taken at a later point in time than the first image;
[0039] c) a third input for receiving a golden truth (GT) indication of a transformation vector specifying a correction of the patient's position and / or orientation, with which correction the dose delivered to the patient optimally approximates the dose originally intended from the first image;
[0040] d) a transformation vector generating unit, which is used to generate a plurality of transformation vectors;
[0041] e) an image transformation unit for obtaining a corresponding transformed second image by applying the transformation vector to the second image;
[0042] f) an adding unit for calculating, for each transformed second image, a corresponding GT correction vector from the GT indication of the second image and the transformation vector used to obtain the transformed second image;
[0043] g) a loss calculation module for determining a loss for each transformed second image based on a difference between a GT correction vector and a correction vector estimated by a dose based control module when the first image and the transformed second image are provided as inputs;
[0044] h) An updating module for updating parameters of the dose-based control module based on the calculated losses.
[0045] The update module is, for example, configured to update the parameters of the dose-based control module through a back-propagation process.
[0046] According to a third aspect of the present disclosure, a method for training a neuromorphic dose-based control module is provided, the method comprising the following operations.
[0047] a) Providing a first image of a patient, said first image comprising tissue labeling data and dose specification data.
[0048] b) providing a second image of the patient, said second image comprising a specified portion of the patient to be treated, together with a gold truth (GT) indication of a transformation vector specifying a correction (xc, yc, zc) of the position and / or orientation of the patient, wherein the dose delivered to the patient optimally approximates the dose originally intended based on the first image.
[0049] c) generating an enhancement set for the second image by applying mutually different geometric transformations in three-dimensional space, such as translations and / or rotations, to the second image, and for each category of the enhancement set for the second image, calculating a golden truth value of the required correction based on the GT indication of the second image and the transformation of the category obtained from the second image.
[0050] d) calculating, using a dose-based control module, a corresponding output vector indicating a correction of the position and / or orientation of the patient for the category of the enhanced set of each pair of the first image and the second image.
[0051] e) For each category of the augmented set of the second image, compute the loss from the difference between the correction specified by the corresponding output vector and the corresponding GT.
[0052] f) Update the parameters of the dose-based control module to reduce losses.
[0053] The training system according to the second aspect and the training method according to the third aspect can effectively train the dose-based control module because for each manually labeled second image, a plurality of transformed second images for training are automatically generated.
[0054] Note that an exemplary embodiment of the improved position correction system including the third control module is configured to update the parameters of the dose-based control module based on a loss defined by the difference between the correction vector determined by the third control module and the correction vector determined by the dose-based control module. The loss defined by the difference is, for example, a distance measure, such as a block-based distance measure (L1), a Euclidean distance measure (L2), or an L∝ distance measure. This allows further training of the dose-based control module to be performed even when the dose-based control module is already in use.
[0055] BRIEF DESCRIPTION OF THE DRAWINGS
[0056] These and other aspects are shown in more detail with reference to the accompanying drawings, in which:
[0057] Figure 1 An embodiment of a system for positioning a patient for treatment in a radiation beam from a radiation source is schematically shown;
[0058] Figure 2 Other embodiments are shown;
[0059] Figure 3 The components of this embodiment are shown;
[0060] Figure 4 further showing exemplary portions of the assembly in detail;
[0061] Figure 5 Schematically illustrates an embodiment of a method for training a neuromorphic dose-based control module;
[0062] Figure 6 An embodiment of a training system for training a dosage-based control module is schematically shown;
[0063] Figure 7 Other embodiments of the positioning system are shown;
[0064] Figure 8 shows exemplary components of the described embodiments in greater detail;
[0065] Fig. 9 Other embodiments of the system are schematically shown;
[0066] Fig. 10A , Fig. 10B , FIG. 11A to FIG. 11D and FIG. 12A to FIG. 12DShows that in training Figure 4 The experimental data obtained in the process of neural network;
[0067] Fig.13 Additional experimental data are shown.
[0068] Detailed description of implementation plan
[0069] Figure 1 Schematically shown is a system 1 for positioning a patient for treatment in a radiation beam from a radiation source 5 (e.g. a photon radiation source or a proton radiation source) based on patient-specific data. The patient-specific data comprises a first image pCT of the patient, the first image pCT comprising tissue label data and dose specification data. Figure 1 As shown, the system includes an imaging device 2, a positioning device 3 and an optimization controller 4.
[0070] A positioning device 3, such as a robotic examination table, is provided for holding the patient in a variable position in the radiation beam from the radiation source 5, so as to perform precise interventions on a designated part of the patient.
[0071] The patient specific data is provided by the oncologist prior to the patient's treatment and typically specifies a series of treatment phases over a period of time.Accordingly, the radiation source 5 is configured to emit a controlled radiation beam having a predetermined distribution over a predetermined time.
[0072] For this, the patient needs to be correctly registered. That is, the patient must be correctly positioned in the radiation beam. For this purpose, the imaging device 2 is configured to provide a second image rCT of the patient, which includes a designated part of the patient to be treated.
[0073] The optimization controller 4 is configured to provide positioning control data ΔP for guiding the positioning device 3 based on the first image pCT and the second image rCT.
[0074] One approach is to register the patient so that the positions of rigid structures within the body, such as bone tissue or landmarks identified in the second image, match those in the first image, pCT.
[0075] However, it is not necessarily sufficient to simply position the patient exactly in the orientation foreseen by the oncologist in preparation for treatment. This is because the distribution of the patient's body tissues may change in the time interval between the date of preparation for treatment and the date of the treatment phase. For example, it may be the case that the patient loses or gains weight or that the size of the tumor being treated decreases or even increases.
[0076] like Figure 2As shown in more detail in , to account for such morphological changes over time, the optimization controller 4 includes a dose-based control module 41 configured to calculate the registration of the patient such that the applied dose optimally matches the dose specified by the oncologist in the treatment plan.
[0077] like Figure 3 As shown, in operation, the dose-based control module 41 performs a neuromorphic data processing operation on the first image data obtained from the first image pCT to present the first feature map data F1. The dose-based control module 41 also performs the same neuromorphic data processing operation on the second image data obtained from the second image rCTq to present the second feature map data F2. The first image pCT and the second image rCTq include voxelized volumetric X-ray data in a DICOM file of a CT modality.
[0078] After these operations, it concatenates the first feature map data F1 and the second feature map data F2 to provide concatenated feature map data Fc. It then performs further neuromorphic data processing operations on the concatenated feature map data Fc to provide a correction vector ΔPd indicating the required repositioning of the patient based on dose optimization.
[0079] In an embodiment, the dosage-based control module 41 is configured to perform the specified operations using a suitably programmed general purpose processor. Figure 3 In an embodiment of the present invention, the dose-based control module 41 includes a first convolutional neuromorphic processing branch 411, a second convolutional neuromorphic processing branch 412 that is identical to the first convolutional neuromorphic processing branch 411, a neuromorphic connection stage 413, and a fully connected neuromorphic stage 414.
[0080] The first convolutional neuromorphic processing branch 411 is configured to receive first image data obtained from the first image pCT, and to provide first feature map data F1 based on the received first image data. The second convolutional neuromorphic processing branch 412 is identical to the first convolutional neuromorphic processing branch 411, and is configured to receive second image data obtained from the second image rCT, and to provide second feature map data F2 based on the received second image data. The neuromorphic concatenation stage 413 is configured to receive the first feature map data F1 and the second feature map data F2, and to provide connected feature map data Fc based on the first feature map data and the second feature map data. The fully connected neuromorphic stage 414 is configured to receive the connected feature map data Fc and to provide an output vector ΔPd indicating a correction in the registration.
[0081] Reference again Figure 2 , in which it is shown that the system 1 for positioning further includes a geometry-based control module 42 and a transformation module 43.
[0082] In operation, the geometry-based control module 42 performs a rigid registration of the geometry to provide a first estimate ΔPg for a desired repositioning of the patient based on the first image pCT and the second image rCT. In the illustrated embodiment, the transformation module 43 is in operation to transform the second image to compensate for the registration difference indicated by the first estimate ΔPg. The dose-based control module 41 is then in operation to determine an estimate ΔPd for an additional repositioning of the patient required based on the first image pCT and the transformed second image rCTg. In the illustrated example, the dose-based control module 41 adds (adder 45) the first estimate ΔPg from the geometry-based control module 42 and the estimate ΔPd from the dose-based control module 41 to calculate positioning control data ΔP for guiding the positioning device.
[0083] In an alternative embodiment, the dose-based control module 41 calculates the positioning control data ΔP directly from the first image pCT and the second image rCT. This has the advantage that the transformation module 43 and the adder 45 are not required, which simplifies the system 1. However, if Figure 2 The embodiment of the system shown has the advantage that the training of the dose-based control module 41 is simplified.
[0084] An exemplary structure of a convolutional neuromorphic processing unit 6 that can be used for both convolutional neuromorphic processing branch 411 and convolutional neuromorphic processing branch 412 is shown in Figure 4 In the illustrated embodiment, the convolutional neuromorphic processing unit 6 includes a plurality of n convolutional neuromorphic processing layers 6_1, ..., 6_n, for example, a few to dozens of layers. In the illustrated example, the selected value of n is equal to 5. In addition, in the illustrated example, the convolutional neuromorphic processing layer includes a 3D convolutional layer 611, a batch normalization layer 612, an activation layer 613 (e.g., a ReLU activation layer (e.g., a leaky ReLU activation layer)), and a 3D pooling layer 614 (e.g., a maximum pooling layer). As an example, the size of the 3D convolution window of the 3D convolutional layer 611 is 3x 3x 3 voxels, and the size of the window used by the 3D pooling layer 614 is 2x 2x 2 voxels. In this example, the window size is the same for all convolutional neuromorphic processing layers, but this is not required. In the example shown, the 3D convolutional layer 611 of the first convolutional neuromorphic processing layer 6_1 has 8 filters, and the 3D convolutional layer 611 of each subsequent convolutional neuromorphic processing layer has twice the number of filters of the previous one. To this end, the convolutional neuromorphic processing unit 6 provides a 128-dimensional feature vector at its output.
[0085] Figure 5 A method for training a neuromorphic dose based control module 41 is schematically illustrated. As shown therein, the method comprises the following operations.
[0086] In step S1 , first volumetric image data, for example planning CT image data pCT of a predetermined planning CT image, is provided to the first convolutional neuromorphic processing branch 411 .
[0087] In step S2, second volumetric image data, e.g. from predetermined repeated CT image data, are provided, and in step S3, an enhanced set {rCT1, ..., rCTn} of predetermined second image data is obtained by applying mutually different transformations {ΔP1, ..., ΔPn} to the predetermined repeated CT images rCT in three-dimensional space. Furthermore, a golden truth value {ΔPg1, ..., ΔPgn} of the required correction for each category of the enhanced set of the second image is calculated from the GT indication of the second image and the transformation applied to this category.
[0088] In other examples, the volumetric image data is synthetic CT (sCT) image data, such as based on CBCT or MRI.
[0089] Then, each of the second image data {rCT1, ..., rCTn} is provided to the convolutional neuromorphic processing branch 412 in step S4.
[0090] As a result, the dose-based control module 41 calculates, for each pair of the first image data pCT and the category rCTi of the enhanced set of the second image, an output vector {ΔPo1, . . . , ΔPon} indicating a correction in the registration.
[0091] In step S5 , for each pair i, the loss Li is calculated from the difference between the correction specified by the output vector ΔPoi calculated by the dose-based control module 41 and the corresponding GTΔPgi of the pair.
[0092] Then in step S6, the weights of the first and second mutually identical convolutional neuromorphic processing branches 411, 412, the neuromorphic connection stage 413 and the fully connected neuromorphic stage 414 are updated, for example using back-propagation, to minimize the loss Li.
[0093] Figure 6 A training system for training the dose-based control module 41 is schematically shown. Figure 6As shown, the training system includes a first input IP for receiving a first image of a patient, in this case, a planning CT image pCT including tissue label data and dose specification data. The training system also includes a second input IR for receiving a second image rCT of the patient. The second image rCT includes a specified portion of the patient to be treated and is taken at a subsequent time point. The oncologist H determines which correction (e.g., position correction (xc, yc, zc) of the patient) is necessary to optimally deliver the dose originally intended for the treatment plan. The transformation vector generation unit 10 generates a plurality of transformations (e.g., translations (xi, yi, zi)) applied to the second image rCT to obtain a transformed second image rCTi through an image transformation unit 12. The correction (xc, yc, zc) is used as the ground truth (GT) value of the correction that is optimal for the second image rCT. Based on the GT value and the known transformation applied to the second image rCT to obtain the translated second image rCTi, the addition unit 14 calculates the corresponding GT value (xcj, ycj, zcj) for each transformed second image rCTi. The trained dose-based control module 41 compares each translated second image rCTj with the first image pCT and estimates the required correction vector (xoj, yoj, zoj). The loss calculation module 16 determines the difference between the estimated correction vector (xoj, yoj, zoj) and the corresponding GT value (xcj, ycj, zcj) and outputs a loss value Lj, for example, a value indicating the distance between the estimated correction vector (xoj, yoj, zoj) and the corresponding GT value (xcj, ycj, zcj) in the space defining the translation vector. The distance measure is, for example, the Manhattan distance, the Euclidean distance, or the L∝ measure. The current network parameters of the dose-based control module 41 are then updated, for example using backpropagation, so as to minimize the loss.
[0094] Figure 7 Other embodiments of the positioning system are shown, the positioning system further comprising a third control module 44, the third control module 44 being configured to calculate a complete dose guidance optimization by applying a gradient descent GD algorithm. The third control module 44 receives an output vector ΔP0 indicating a registration correction calculated by the dose-based control module 41. In an alternative embodiment, the input to the third control module 44 is as follows Figure 2 The output ΔP is calculated in the embodiment shown.
[0095] Figure 8An embodiment of the third control module 44 is shown in more detail. As shown in this embodiment, a transformation vector modifier 440 is provided, which generates a set of modified transformation vectors {ΔPj1, ..., ΔPjn} for each transformation vector ΔPj at its input. Thus, for a transformation vector ΔP0 received from the dose-based control module 41, it generates a set of modified transformation vectors {ΔP01, ..., ΔP0n}. The image transformation module 441 provides a set of transformed second image data {rCTj1, ..., rCTjn}. Each category of modified second image data rCTjk is obtained by transforming the second image data (rCT) according to the corresponding modified transformation vector ΔPjk.
[0096] Based on the predetermined radiation therapy plan (RT plan), the dose simulation module 442 then provides an estimated spatial dose distribution Djk for each modified repeated CT image data rCTjk. The simulation module 442 estimates the spatial dose distribution Djk using, for example, Monte Carlo simulation.
[0097] For each estimated spatial dose distribution Djk, the treatment effect calculation module 443 calculates an indication NTCPjk of the effect of the estimated spatial dose distribution on the patient. The calculation takes into account the RT structure information and the information on the prognostic factors.
[0098] Furthermore, the dose volume histogram (444) calculation module calculates an expected dose volume histogram DVHjk for each estimated spatial dose distribution Djk taking into account the RT structure information.
[0099] The evaluation module 445 calculates an overall quality measure Qjk indicating the expected quality Qjk of the treatment based on the calculated indications NTCPjk and DVHjk for the estimated spatial dose distribution Djk.
[0100] The gradient descent module 446 calculates a single next-iteration transformation vector ΔPj+1. To do this, the gradient descent module 446 determines which direction in the space in which the transformation vectors are defined leads to the highest improvement in expected quality.
[0101] As an example, where the transformation vector ΔPj includes components (Δxj, Δyj, Δzj), the transformation vector modifier 440 may generate a set of modified transformation vectors {ΔPj1, ..., ΔPjn}, where the modified transformation vectors have components (Δxj+dxk, Δyj+dyk, Δzj+dzk), where the displacements dxk, dyk, dzk are uniformly distributed in a volume centered at the origin in the space of transformation vectors. The gradient descent module 446 may then select one of the modified transformation vectors ΔPjk corresponding to the highest value of Qjk as the next iterative transformation vector ΔPj+1.
[0102] In the above embodiments, it is assumed that the transformation vector space is a three-dimensional space. In this case, the transformation vector is a three-dimensional vector in mutually orthogonal spatial directions x, y, z. In these embodiments, the dose-based control module (41) is configured to provide positioning control data ΔP to the three-dimensional control vector, and the three-dimensional control vector causes the positioning device to take a corrected position in the three-dimensional space. In practice, significant improvements have been achieved in this regard. In other embodiments, the transformation vector is a higher-dimensional vector that includes not only components of the spatial directions x, y, z, but also one or more components related to the direction in space. In these other embodiments, the dose-based control module 41 is not only configured to provide positioning control data to cause the positioning device to take a corrected position in the three-dimensional space, but the dose-based control module 41 is also configured to provide direction control data to cause the positioning device to take a corrected direction in the three-dimensional space. In this regard, further improvements in dose control are possible.
[0103] Fig. 9 Other embodiments of the system are schematically shown. Figure 2 In an embodiment of the system, a geometry-based control module 42 is included. In this embodiment, the geometry-based control module 42 provides its calculated correction vector ΔPg to the input of the dose-based control module 42 and to the input of the other control module 44. Based on the correction vector ΔPg provided at their respective inputs, the dose-based control module 42 and the other control module 44 each calculate a correction vector ΔP0, ΔPg, respectively, for dose-based positioning. A quality assessment module 46, further included in this embodiment of the system, compares the correction vector ΔPg calculated by the other control module 44 with the correction vector ΔP0 calculated by the dose-based control module 42. Based on the comparison, the quality assessment module 46 determines a final correction vector ΔPf, and the system positions the patient according to the final correction vector ΔPf thus obtained.
[0104] Experimental Results
[0105] In the first experiment, for the purpose of providing a geometry-based correction vector, we trained Figure 3 As shown and referenced in Figure 3 Describe the neural network.
[0106] The first test was performed with one patient image pair (pCT+rCT). The rCT image was enhanced 1000 times by applying a synthetic shift of -10 cm to 10 cm in x, y and z directions.
[0107] The results of the training process are shown in Fig. 10A and Fig. 10B middle.
[0108] Fig. 10A and Fig. 10B The training loss expressed as mean squared error (MSE) and mean absolute error (MAE) are shown as a function of the number of training cycles, respectively. After training, the neural network estimates geometry-based correction vectors with a mean absolute error of less than 1 mm for all axes.
[0109] In the second experiment, a second training / validation set was used, in which data from 45 patients were used. Each patient had one pCT and 5 repeated CTs, totaling 225 image pairs. These images were then enhanced 5 times with a synthetic shift of -5 cm to +5 cm in the x, y, and z axes. The total number of images used was 1125 images.
[0110] The results of the training process are shown in FIG. 11A to FIG. 11D middle.
[0111] Fig.11A and Fig. 11B The training loss expressed as mean squared error (MSE) and mean absolute error (MAE) during a short training period (100 epochs) is shown as a function of the number of training epochs, respectively.
[0112] Fig. 11C and Fig.11D The training loss expressed as mean square error (MSE) and mean absolute error (MAE) during a long training period (600 epochs) is shown as a function of the number of training epochs, respectively. After the long training period is completed, the neural network estimates geometry-based correction vectors with a mean absolute error of less than 3 mm for all axes.
[0113] In the third experiment, for the purpose of providing a dose-based correction vector, the training Figure 3 As shown and referenced in Figure 3 The neural network described in
[24] is presented in Figure 2. In this case, the daily dose is first estimated at the current patient position in the rCT image. Then, the network is trained with the planned dose and the daily dose in order to optimize the translation offset previously obtained by gradient descent.
[0114] The results of the training process are shown in FIG. 12A to FIG. 12D middle.
[0115] Fig. 12A and Fig. 12B The training loss expressed as mean squared error (MSE) and mean absolute error (MAE) during a short training period (30 epochs) is shown as a function of the number of training epochs, respectively.
[0116] Fig. 12C and Fig.12DThe training loss expressed as mean square error (MSE) and mean absolute error (MAE) during a long training period (80 epochs) is shown as a function of the number of training epochs, respectively. After the long training period is completed, the neural network estimates dose-based correction vectors with a mean absolute error of less than 1.2 mm for all axes. In any case, the relatively modest number of training examples is sufficient for a network that has been trained to compute geometry-based corrections, since training does not need to start from scratch.
[0117] Fig.13 A comparison of the speed of calculations involved in three approaches to achieve dose-based optimization is shown. The comparison was performed on 10 patients with head and neck cancer who had previously received proton radiotherapy at GPTC. For each patient, these calculations required imaging data (planning CT and repeat CT, segmentation files) and treatment data (radiotherapy plan) in DICOM format.
[0118] The three methods include:
[0119] 1) RS-MC dose engine from RaySearch's treatment planning system (TPS) software called RayStation.
[0120] 2) Gradient descent methods when using a fast Monte Carlo code called gPMC, which has been developed at Massachusetts General Hospital (MGH). This is a GPU-based dose engine for proton therapy that focuses only on dose and LET calculations.
[0121] 3) As reference Figure 4 Specifies the computation to be performed by the CNN.
[0122] For each patient, the left column shows the computation time involved.
[0123] Calculations according to Method 1 and Method 2 were performed using a GPU-based clinical RayStation client configuration, which included:
[0124] o 12 physical core dual-socket Intel 2.6GHz CPU
[0125] o NVIDIA RTX8000 GPU
[0126] o 48GB DDR3 ECC RAM
[0127] o 2x 150SSD in RAID 1
[0128] CNN-based optimization is implemented on GPTC’s clinical QA platform server Figure 4The CNN is conducted, which includes:
[0129] o 24 physical cores dual-socket Intel 3.0GHz CPU
[0130] o 2x NVIDIA GTX 1080 GPUs
[0131] o 64GB DDR4 RAM
[0132] o 4x 250GB SSD
[0133] Fig.13 The computation time involved for each patient for method 1), method 2), and method 3) respectively is shown.
[0134] The results are summarized in the table below.
[0135] method Tmin(min) Tmax(min) Tav(min) RS-MC 32 65 45 g 4 8 6 CNN 1 1 1
[0136] In summary, it can be seen that a very fast dose-based optimization is achieved with the CNN-based method (3). Even though the CNN-based calculations are performed with a GPU-based clinical RayStation client configuration, it is still significantly faster than the RS-MC-based method. The optimization results obtained by this method (3) can be independently verified with the gMC-based method (2), which is also significantly faster than the RS-MC-based method. In this regard, the verified optimization results can also be achieved significantly faster than the RS-MC-based method.
Claims
1. A system (1) for positioning a patient for treatment in a radiation beam based on patient-specific data, the patient-specific data comprising a first image (pCT) of the patient, the first image of the patient comprising tissue label data and dose specification data, the system comprising: an imaging device (2) configured to provide a second image (rCT) of the patient, the second image of the patient comprising a designated portion of the patient to be treated; a positioning device (3) for maintaining the patient in a variable position and / or orientation in the radiation beam for precise intervention on a specific part of the patient; and an optimization controller (4) configured to provide registration control data (ΔP) to guide the positioning device (3) based on the first image (pCT) and the second image (rCT); wherein the optimization controller (4) comprises a dose-based control module (41), the dose-based control module (41) being configured to calculate an optimized registration of the patient by performing a dose-guided optimization, The dosage-based control module is configured to: performing a neuromorphic data processing operation on first image data obtained from said first image (pCT) to render first feature map data (F1); performing said neuromorphic data processing operation on second image data obtained from said second image (rCT) to render second feature map data (F2); concatenating the first feature map data (F1) and the second feature map data (F2) to provide concatenated feature map data (Fc); performing further neuromorphic data processing operations on the concatenated feature map data (Fc) to provide a correction vector indicative of a desired registration for the patient based on dose optimization, Characterized in that the system comprises a further control module (44) configured to calculate an optimized correction vector for dose guidance by applying a gradient descent (GD) algorithm, The other control module (44) comprises: a transformation vector modifier (440) for generating, at its input, for each transformation vector (ΔPj), a set of modified transformation vectors ({ΔPj1, ..., ΔPjn}); an image transformation module (441) for transforming the second image (rCT) according to a corresponding modified transformation vector (ΔPjk) in the set of modified transformation vectors ({ΔPj1, ..., ΔPjn}) to provide a set of modified second images ({rCTj1, ..., rCTjn}); a dose simulation module (442) for providing an estimated spatial dose distribution (Djk) for each modified second image (rCTjk); A dose volume histogram (444) calculation module, which is used to calculate an expected dose volume histogram (DVHjk) generated by each estimated spatial dose distribution (Djk); an evaluation module (445) for calculating a quality measure (Qjk) indicative of an expected quality (Qjk) of the treatment based on the calculated expected dose volume histogram (DVHjk) for each estimated spatial dose distribution (Djk); A gradient descent module (446) for calculating a single next iteration transformation vector (ΔPj+1) based on respective values of expected qualities (Qjk) determined for modified transformation vectors (ΔPjk) in the set of modified transformation vectors ({ΔPj1, ..., ΔPjn}).
2. The system (1) according to claim 1, further comprising: a geometry-based control module (42) for performing a geometric rigid registration to provide a first estimate (ΔPg) of a required registration correction for the patient based on the first image (pCT) and the second image (rCT), a transformation module (43) for transforming the first image or the second image to compensate for the registration difference indicated by the first estimate (ΔPg), The dose-based control module (41) is configured to determine an estimate (ΔPd) of a required additional registration correction for the patient based on the first image (pCT) and the second image (rCT), wherein one of the first image (pCT) and the second image (rCT) is transformed.
3. The system according to claim 1 or 2, wherein the dosage-based control module (41) comprises: a first convolutional neuromorphic processing branch (411) configured to receive first image data obtained from said first image (pCT) and to provide said first feature map data (F1) based on said received first image data, a second convolutional neuromorphic processing branch (412), which is identical to the first convolutional neuromorphic processing branch (411) and is configured to receive second image data obtained from the second image (rCT) and to provide the second feature map data (F2) based on the received second image data, - a neuromorphic concatenation stage (413) configured to receive the first feature map data (F1) and the second feature map data (F2) and to provide the concatenated feature map data (Fc) based on the first feature map data and the second feature map data; - a fully connected neuromorphic stage (414) configured to receive said concatenated feature map data (Fc) and to provide a correction vector (ΔPd) indicative of a correction in said registration.
4. The system according to claim 3, wherein the mutually identical convolutional neuromorphic processing branches (411, 412) each comprise a plurality of stages (6_1, ..., 6_n) having a convolutional neuromorphic layer (611).
5. The system of claim 4, wherein each stage (6_1, ..., 6_n) of the convolutional neuromorphic processing branch comprises a batch normalization layer (612).
6. A system according to claim 4 or 5, wherein each stage (6_1, ..., 6_n) of the convolutional neuromorphic processing branch comprises a leaky ReLU activation layer (613).
7. A system according to claim 4, 5 or 6, wherein each stage (6_1, ..., 6_n) of the convolutional neuromorphic processing branch comprises a pooling layer (614).
8. The system of claim 5, wherein the pooling layer (614) is a max pooling layer.
9. The system of any of the preceding claims, wherein the simulation module (442) estimates the spatial dose distribution (Djk) using Monte Carlo simulation.
10. A system according to any of the preceding claims, further comprising a treatment effect calculation module (443), which takes into account one or more of the RT structural information and the information about the prognostic factors to provide an estimate of the normal tissue complication probability (NTCPjk) associated with each estimated spatial dose distribution (Djk) on the human body, wherein the evaluation module (445) is configured to calculate the quality measure (Qjk) based on the calculated expected dose volume histogram (DVHjk) and based on the estimate of the probability.
11. A system according to any of the preceding claims, configured to update parameters of the dose-based control module (41) based on a loss defined by a difference between a correction vector determined by the third control module (44) and a correction vector determined by the dose-based control module (41).
12. A system according to any of the preceding claims, wherein the further control module (44) is configured to receive at its input a correction vector (ΔP0) provided by the dose-based control module (41), and wherein the system is configured to position the patient according to the correction vector calculated by the further control module.
13. A system according to claim 2, wherein the other control module (44) is configured to receive at its input a correction vector (ΔPg) provided by the geometry-based control module (42), the system further comprising a quality assessment module (46) to compare the correction vector calculated by the other control module (44) with the correction vector calculated by the dose-based control module (42), wherein the system is configured to position the patient according to the correction vector (ΔPf) determined by the quality assessment module (46) based on the comparison.
14. A training system for training a dose-based control module (41), comprising: a first input (IP) for receiving a first image (pCT) of a patient, said first image comprising tissue labeling data and dose specification data; a second input (IR) for receiving a second image (rCT) of the patient, the second image (rCT) comprising a designated portion of the patient to be treated and taken at a later point in time than the first image; a third input (H) for receiving a ground truth (GT) indication of a transformation vector specifying a correction (xc, yc, zc) of the position and / or orientation of said patient, with which correction the dose delivered to said patient optimally approximates the dose originally intended from said first image; A transformation vector generating unit (10), which is used to generate a plurality of transformation vectors (xi, yi, zi); an image transformation unit (12) for obtaining a corresponding transformed second image (rCTi) by applying the transformation vector (xi, yi, zi) to the second image (rCT); an adding unit (14) for calculating, for each transformed second image (rCTi), a corresponding GT correction vector (xcj, ycj, zcj) from a GT indication of said second image (rCT) and a transformation vector (xi, yi, zi) for obtaining said transformed second image (rCTi); a loss calculation module (16) for determining a loss for each transformed second image based on a difference between the corresponding GT correction vector and a correction vector (xoj, yoj, zoj) estimated by the dose-based control module when the first image and the transformed second image are provided as input; An updating module (18) for updating parameters of the dose-based control module (41) based on the calculated loss.
15. A method for training a neuromorphic dose-based control module (41), comprising: Providing (S1) a first image (pCT) of a patient, the first image (pCT) comprising tissue label data and dose specification data; providing (S2) a second image (rCT) of the patient, the second image (rCT) comprising a specified portion of the patient to be treated, together with a ground truth (GT) indication of a transformation vector specifying a correction (xc, yc, zc) of the position and / or orientation of the patient, with which correction the dose delivered to the patient optimally approximates the dose originally intended based on the first image; generating (S3) an enhanced set of second images ({rCT1, ..., rCTn}) by applying mutually different geometric transformations {ΔP1, ..., ΔPn} in three-dimensional space to the second image (rCT), and for each category of the enhanced set of the second image, calculating the ground truth value of the required correction according to the GT indication of the second image and the transformation applied to the category; calculating (S4) using the dose-based control module (41) a corresponding output vector {ΔPo1, ..., ΔPon} indicating a correction (xc, yc, zc) of the position and / or orientation of the patient for each class (rCTi) of the enhancement set ({rCT1, ..., rCTn}) of the first image (pCT) and the second image; for each category (rCTi) of the set of enhancements ({rCT1, ..., rCTn}) of the second image, computing (S5) a loss (Li) from the difference between the correction specified by the corresponding output vector and the corresponding GT; The parameters of the dosage-based control module (41) are updated (S6) to reduce the loss (Li).
