A Smart Radiotherapy Planning Method and System

Abstract

This invention discloses an intelligent radiotherapy planning method and system. The intelligent radiotherapy planning method includes: inputting the patient's radiotherapy plan input information; predicting the patient's three-dimensional dose distribution using a dose prediction model; generating an initial radiotherapy plan based on the patient's radiotherapy plan input information and the three-dimensional dose distribution; and automatically learning and optimizing the parameters of the initial radiotherapy plan using a deep Q-network model to obtain a final radiotherapy plan that meets clinical requirements. This intelligent radiotherapy planning method of the present invention achieves automatic prediction of the three-dimensional dose distribution through a dose prediction model, and simultaneously automatically optimizes the parameters of the initial radiotherapy plan based on the three-dimensional dose distribution using a deep Q-network model, ultimately automatically generating a final radiotherapy plan that meets clinical requirements. Therefore, this intelligent radiotherapy planning method of the present invention establishes a model combining dose prediction and automatic parameter optimization, realizing automatic, accurate, and efficient radiotherapy plan optimization, and has significant practical value.

Description

Technical Field

[0001] This invention relates to the field of radiotherapy planning technology, and more specifically to a smart radiotherapy planning method and system. Background Technology

[0002] In the radiotherapy industry, a radiotherapy plan is a document that controls how an accelerator delivers radiation. Just as the human brain controls human behavior, how an accelerator treats a patient is controlled through the radiotherapy plan. The radiotherapy plan determines the patient's treatment outcome and the severity of subsequent side effects. Once the radiotherapy plan is finalized, it is rarely modified during the entire treatment period unless the patient's irradiation range needs to be changed midway through treatment; therefore, the radiotherapy plan is crucial.

[0003] Existing radiotherapy planning systems rely heavily on manual adjustment of target area and organ-at-risk parameters, resulting in low levels of intelligence and automation. Therefore, a proposed application (application number 202211144237.8) entitled "An Automated Planning Method and System Based on Dose Prediction and Parameter Optimization" describes a method that includes: performing a planned CT scan on the patient to obtain CT images and generating organ delineation data from these images; inputting the CT images and organ delineation data into a trained 3D VGG-U-Net network model to predict the three-dimensional dose distribution of organs at risk, and obtaining the average dose of these organs based on the three-dimensional dose distribution prediction; determining the objective function of the irradiation field distribution model based on the organ delineation data and the average dose of these organs to design an initial radiotherapy plan, and optimizing the parameters of the objective function using a hybrid particle swarm optimization and genetic algorithm to obtain parameter optimization results; evaluating the parameter optimization results, updating the parameters of the objective function based on the evaluation results, and solving the objective function based on the updated parameter results to obtain the optimal radiotherapy plan. This invention enables intelligent algorithms to automatically optimize parameters in radiotherapy planning, thereby improving the quality and efficiency of the planning process.

[0004] The aforementioned invention patent optimizes the objective function by using a hybrid optimization algorithm of particle swarm optimization and genetic optimization after dose prediction. However, the invention patent also has technical defects such as excessively high computational complexity of the algorithm and the fact that it only utilizes the average dose information of organs at risk in predicting the three-dimensional dose distribution.

[0005] In view of this, this invention patent is hereby proposed. Summary of the Invention

[0006] To address the above problems, this invention provides a method and system for intelligent radiotherapy planning, specifically employing the following technical solution:

[0007] A smart radiotherapy planning method, comprising:

[0008] Input the patient's radiotherapy plan information, and predict the patient's three-dimensional dose distribution through a dose prediction model;

[0009] An initial radiotherapy plan is generated based on the patient's radiotherapy plan input information and three-dimensional dose distribution.

[0010] By using a deep Q-network model to automatically learn and optimize the parameters of the initial radiotherapy plan, a final radiotherapy plan that meets clinical requirements is obtained.

[0011] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the automatic learning parameter optimization of the initial radiotherapy plan using a deep Q-network model includes:

[0012] DVH curves were obtained by three-dimensional dose distribution, and key ROIs and their corresponding dose-volume constraints were extracted from the DVH curves.

[0013] Adjustment actions are set for the dose-volume constraints of each key ROI: limit increase by a fixed percentage, limit remain unchanged, limit decrease by a fixed percentage, and weight increase by a fixed percentage, weight remain unchanged, and weight decrease by a fixed percentage, to obtain a discrete action set.

[0014] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the automatic learning parameter optimization of the initial radiotherapy plan is performed using a deep Q-network model, wherein the deep Q-network model includes:

[0015] A value function q is constructed using an embedded dose prediction model. * The estimated model Q(s,a;W) is given by (s,a), where s represents the state, a represents the action, and W represents the model parameters.

[0016] The estimation model Q(s,a;W) includes:

[0017] a) Input: The input is the dose distribution. The initial input is the three-dimensional dose distribution predicted by the dose prediction model, and the subsequent input is the planned optimized three-dimensional dose distribution.

[0018] b) Embedding layer, represented by a dose prediction model;

[0019] c) Linear layer: Stack several linear layers to obtain the output, which is 1 × the size of the discrete action set, representing the Q value of each action. Select the action with the largest Q value as the current action.

[0020] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the automatic learning parameter optimization of the initial radiotherapy plan is performed using a deep Q-network model, wherein the deep Q-network model includes:

[0021] Define the scoring function For each key ROI, the score is defined as a continuous function between 0 and 1. The sum of the scores is then used to obtain the reward function.

[0022] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the automatic learning parameter optimization of the initial radiotherapy plan is performed using a deep Q-network model, wherein the deep Q-network model includes:

[0023] The loss function is defined by combining empirical replay and a fixed Q-target:

[0024]

[0025] Where γ is the discount factor, the specific implementation of experience replay involves maintaining a replay buffer. Each time a four-tuple (state, action, reward, next state) is sampled from the environment, it is stored in the replay buffer. When training the deep Q-network model, several samples are randomly drawn from the replay buffer for training. Simultaneously, the deep Q-network model... The network parameters are kept relatively constant during training and updated to the latest parameters at regular training intervals.

[0026] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the automatic learning parameter tuning of the initial radiotherapy plan using a deep Q-network model includes:

[0027] Historical patient tumor precision radiotherapy planning data were collected and divided into training set and validation set. The training set included CT data and key ROI delineation data, while the validation set included the final radiotherapy plan.

[0028] The trained dose prediction model is used to predict the three-dimensional dose distribution on the training set data.

[0029] DVH curves are obtained through three-dimensional dose distribution. Key ROIs and their corresponding dose-volume constraints are extracted from the DVH curves. Adjustment actions are set for the dose-volume constraints of each key ROI to obtain a discrete action set.

[0030] A value function q is constructed using an embedded dose prediction model. * The estimated model Q(s,a;W) of (s,a) has the initial input of the three-dimensional dose distribution predicted by the MB-UNet model, and the subsequent input is the three-dimensional dose distribution to be optimized. The output is: 1× discrete action set size, representing the Q value of each action, and the action with the largest Q value is selected as the current action.

[0031] The scoring function is defined using a piecewise function. For each key ROI, the score is defined as between 0 and 1, with the segment threshold set by a senior physicist. The scores are summed to obtain the reward function.

[0032] The model is trained by combining empirical replay and a fixed Q-target loss function. Where γ is the discount factor, and during the training of the deep Q-network model, a number of samples are randomly drawn from the replay buffer for training. Simultaneously, the deep Q-network model's... The network parameters are kept relatively constant during training and updated to the latest parameters every preset number of epochs.

[0033] During deep Q-network model training, a maximum number of iterations is set for each patient's data set, and the maximum number of iterations is completed sequentially for each training set: that is, initializing scenario s. 0 For the three-dimensional dose distribution predicted by the dose prediction model, action a is selected using the ∈-greedy strategy. 0 Using a 0 The constraints are updated, and then the radiation therapy planning system optimization interface is invoked to obtain the dose distribution, i.e., the next scenario s. 1 Calculate the reward function r 0 Finally, the sample (s) is stored in the playback buffer. 0 ,a 0 ,r 0 ,s 1 Finally, the parameters of the deep Q-network model are trained using the experience replay strategy.

[0034] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the step of automatically learning and optimizing the parameters of the initial radiotherapy plan using a deep Q-network model to obtain a final radiotherapy plan that meets clinical requirements includes:

[0035] After the deep Q-network model has been trained, a new patient's data is input;

[0036] The three-dimensional dose distribution is predicted using a dose prediction model, and action a is selected using a trained deep Q-network model. 0 Thus, the reward function r is obtained. 0 Continue, and obtain the reward function r. 1 ,r 2 …r n When n reaches the set value, the optimization ends and the final plan is obtained.

[0037] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the radiotherapy plan input information of the patient includes: patient CT scan, target area and key ROI delineation, and the radiotherapy plan input information is standardized.

[0038] After the radiotherapy planning input information is standardized, all radiotherapy planning input information, distance features extracted from the radiotherapy planning input information, and beam mask features extracted from the target area are converted into fixed sizes and then combined into the input of the dose prediction model.

[0039] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the dose prediction model uses a multi-branch UNet model (MB-UNet), which has a "U"-shaped structure and includes an encoding layer, a decoding layer and a skip connection layer.

[0040] This invention also provides a radiotherapy planning system for implementing the aforementioned intelligent radiotherapy planning method, comprising:

[0041] The preprocessing and input module is used to input the patient's radiotherapy plan information.

[0042] The dose prediction module predicts the patient's three-dimensional dose distribution through a dose prediction model and generates an initial radiotherapy plan based on the patient's radiotherapy plan input information and the three-dimensional dose distribution.

[0043] The automatic parameter optimization module uses a deep Q-network model to automatically learn and fine-tune the parameters of the initial radiotherapy plan, resulting in a final radiotherapy plan that meets clinical requirements.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] This invention provides an intelligent radiotherapy planning method that automatically predicts the three-dimensional dose distribution using a dose prediction model. Simultaneously, it automatically optimizes the parameters of the initial radiotherapy plan based on the three-dimensional dose distribution using a deep Q-network model, ultimately generating a final radiotherapy plan that meets clinical requirements. Therefore, this intelligent radiotherapy planning method establishes a model combining dose prediction and automatic parameter optimization, achieving automatic, accurate, and efficient radiotherapy plan optimization, and has significant practical value. Attached image description:

[0046] Figure 1 In Embodiment 1 of this invention, an embedded MB-UNet model is used to construct the value function q. * Framework diagram of the estimation model Q(s,a;W) for (s,a);

[0047] Figure 2Example diagram of the score distribution of each key ROI in Embodiment 1 of the present invention;

[0048] Figure 3 A schematic diagram comparing the automatic plan with the original plan in Embodiment 1 of the present invention;

[0049] Figure 4 A block diagram of a radiotherapy intelligent planning system according to an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of the principle framework of the dose prediction model in Embodiment 2 of the present invention;

[0051] Figure 6 A specific example diagram of the hybrid module in the dose prediction model of Embodiment 2 of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention.

[0053] Therefore, the following detailed description of embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely illustrates some embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0054] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the present invention can be combined with each other.

[0055] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0056] In the description of this invention, it should be noted that the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. These terms are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. In addition, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0057] See Figure 1As shown, a method for intelligent radiotherapy planning in this embodiment includes:

[0058] Input the patient's radiotherapy plan information, and predict the patient's three-dimensional dose distribution through a dose prediction model;

[0059] An initial radiotherapy plan is generated based on the patient's radiotherapy plan input information and three-dimensional dose distribution.

[0060] By using a deep Q-network model to automatically learn and optimize the parameters of the initial radiotherapy plan, a final radiotherapy plan that meets clinical requirements is obtained.

[0061] This embodiment of an intelligent radiotherapy planning method automatically predicts the three-dimensional dose distribution using a dose prediction model. Simultaneously, it automatically optimizes the parameters of the initial radiotherapy plan based on the three-dimensional dose distribution using a deep Q-network model, ultimately automatically generating a final radiotherapy plan that meets clinical requirements. Therefore, this intelligent radiotherapy planning method establishes a model combining dose prediction and automatic parameter optimization, achieving automatic, accurate, and efficient radiotherapy plan optimization, and has significant practical value.

[0062] As an optional implementation of this embodiment, in a radiotherapy intelligent planning method of this embodiment, the automatic learning parameter optimization of the initial radiotherapy plan using a deep Q-network model includes:

[0063] DVH curves are obtained through three-dimensional dose distribution, and key ROIs and their corresponding dose-volume constraints are extracted from the DVH curves.

[0064] Adjustment actions are set for the dose-volume constraints of each key ROI: increase the limit by a fixed percentage, keep the limit unchanged, decrease the limit by a fixed percentage, increase the weight by a fixed percentage, keep the weight unchanged, decrease the weight by a fixed percentage, and obtain a discrete action set.

[0065] Furthermore, in this embodiment, the automatic learning parameter tuning of the initial radiotherapy plan using a deep Q-network model is described, wherein the deep Q-network model includes:

[0066] A value function q is constructed using an embedded dose prediction model. * The estimated model Q(s,a;W) is given by (s,a), where s represents the state, a represents the action, and W represents the model parameters.

[0067] The estimation model Q(s,a;W) includes:

[0068] a) Input: The input is the dose distribution. The initial input is the three-dimensional dose distribution predicted by the dose prediction model, and the subsequent input is the planned optimized three-dimensional dose distribution.

[0069] b) Embedding layer, represented by a dose prediction model;

[0070] c) Linear layer: Stack several linear layers to obtain the output, which is 1 × the size of the discrete action set, representing the Q value of each action. Select the action with the largest Q value as the current action.

[0071] Furthermore, in this embodiment, the automatic learning parameter tuning of the initial radiotherapy plan using a deep Q-network model is described, wherein the deep Q-network model includes:

[0072] Define the scoring function For each key ROI, the score is defined as a continuous function between 0 and 1. The sum of the scores is then used to obtain the reward function.

[0073] Furthermore, in this embodiment, the automatic learning parameter tuning of the initial radiotherapy plan using a deep Q-network model is described, wherein the deep Q-network model includes:

[0074] The loss function is defined by combining empirical replay and a fixed Q-target:

[0075]

[0076] Where γ is the discount factor, the specific implementation of experience replay involves maintaining a replay buffer. Each time a four-tuple (state, action, reward, next state) is sampled from the environment, it is stored in the replay buffer. When training the deep Q-network model, several samples are randomly drawn from the replay buffer for training. Simultaneously, the deep Q-network model... The network parameters are kept relatively constant during training and updated to the latest parameters at regular training intervals.

[0077] As an optional implementation of this embodiment, the automatic learning parameter tuning of the initial radiotherapy plan using a deep Q-network model is described in this embodiment. The training process of the deep Q-network model includes:

[0078] Historical patient tumor precision radiotherapy planning data were collected and divided into training set and validation set. The training set included CT data and key ROI delineation data, while the validation set included the final radiotherapy plan.

[0079] The trained dose prediction model is used to predict the three-dimensional dose distribution on the training set data.

[0080] DVH curves are obtained through three-dimensional dose distribution. Key ROIs and their corresponding dose-volume constraints are extracted from the DVH curves. Adjustment actions are set for the dose-volume constraints of each key ROI to obtain a discrete action set.

[0081] A value function q is constructed using an embedded dose prediction model. * The estimated model Q(s,a;W) of (s,a) has the initial input of the three-dimensional dose distribution predicted by the MB-UNet model, and the subsequent input is the three-dimensional dose distribution to be optimized. The output is: 1× discrete action set size, representing the Q value of each action, and the action with the largest Q value is selected as the current action.

[0082] The scoring function is defined using a piecewise function. For each key ROI, the score is defined as between 0 and 1, with the segment threshold set by a senior physicist. The scores are summed to obtain the reward function.

[0083] The model is trained by combining empirical replay and a fixed Q-target loss function. Where γ is the discount factor, and during the training of the deep Q-network model, a number of samples are randomly drawn from the replay buffer for training. Simultaneously, the deep Q-network model's... The network parameters are kept relatively constant during training and updated to the latest parameters every preset number of epochs.

[0084] During deep Q-network model training, a maximum number of iterations is set for each patient's data set, and the maximum number of iterations is completed sequentially for each training set: that is, initializing scenario s. 0 For the three-dimensional dose distribution predicted by the dose prediction model, action a is selected using the ∈-greedy strategy. 0 Using a 0 The constraints are updated, and then the radiation therapy planning system optimization interface is invoked to obtain the dose distribution, i.e., the next scenario s. 1 Calculate the reward function r 0 Finally, the sample (s) is stored in the playback buffer. 0 ,a 0 ,r 0 ,s 1 Finally, the parameters of the deep Q-network model are trained using the experience replay strategy.

[0085] In this embodiment of the intelligent radiotherapy planning method, the step of automatically learning and optimizing the parameters of the initial radiotherapy plan using a deep Q-network model to obtain a final radiotherapy plan that meets clinical requirements includes:

[0086] After the deep Q-network model has been trained, a new patient's data is input;

[0087] The three-dimensional dose distribution is predicted using a dose prediction model, and action a is selected using a trained deep Q-network model. 0Thus, the reward function r is obtained. 0 Continue, and obtain the reward function r. 1 ,r 2 …r n When n reaches the set value, the optimization ends and the final plan is obtained.

[0088] In a radiotherapy intelligent planning method of this embodiment, the radiotherapy plan input information for inputting the patient includes: patient CT scan, target area and key ROI delineation, and the radiotherapy plan input information is standardized.

[0089] After the radiotherapy planning input information is standardized, all radiotherapy planning input information, distance features extracted from the radiotherapy planning input information, and beam mask features extracted from the target area are converted into fixed sizes and then combined into the input of the dose prediction model.

[0090] The standardization process includes: resampling, dose alignment, cropping and patching, shrinking and expanding, and calculating the distance matrix.

[0091] In this embodiment of a radiotherapy intelligent planning method, the dose prediction model uses a multi-branch UNet model (MB-UNet), which has a "U"-shaped structure and includes an encoding layer, a decoding layer, and a skip connection layer.

[0092] See Figure 4 As shown, this embodiment also provides a radiotherapy planning system for implementing the intelligent radiotherapy planning method, including:

[0093] The preprocessing and input module is used to input the patient's radiotherapy plan information.

[0094] The dose prediction module predicts the patient's three-dimensional dose distribution through a dose prediction model and generates an initial radiotherapy plan based on the patient's radiotherapy plan input information and the three-dimensional dose distribution.

[0095] The automatic parameter optimization module uses a deep Q-network model to automatically learn and fine-tune the parameters of the initial radiotherapy plan, resulting in a final radiotherapy plan that meets clinical requirements.

[0096] This embodiment also provides a computer-readable storage medium storing a computer-executable program, which, when executed, implements the intelligent radiotherapy planning method as described above.

[0097] The computer-readable storage medium described in this embodiment may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0098] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory is used to store a computer-executable program, and when the computer program is executed by the processor, the processor executes the aforementioned intelligent radiotherapy planning method.

[0099] The electronic device is manifested in the form of a general-purpose computing device. It may contain one or more processors that work collaboratively. This invention also does not preclude distributed processing, meaning that processors may be distributed across different physical devices. The electronic device of this invention is not limited to a single entity, but may also be the sum of multiple physical devices.

[0100] The memory stores a computer-executable program, typically machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least some steps of the method.

[0101] The memory includes volatile memory, such as random access memory (RAM) and / or cache memory, and may also be non-volatile memory, such as read-only memory (ROM).

[0102] It should be understood that the electronic device of the present invention may also include elements or components not shown in the examples above. For example, some electronic devices also include display units such as a display screen, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. Any electronic device capable of executing a computer-readable program in its memory to implement the method of the present invention or at least some steps of the method can be considered as an electronic device covered by the present invention.

[0103] From the above description of the embodiments, those skilled in the art will readily understand that the present invention can be implemented by hardware capable of executing specific computer programs, such as the system of the present invention, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. included in the system. The present invention can also be implemented by computer software that executes the methods of the present invention, for example, by control software executed by a microprocessor, electronic control unit, client, server, etc. However, it should be noted that the computer software executing the methods of the present invention is not limited to execution in one or a specific set of hardware entities; it can also be implemented in a distributed manner by unspecified hardware. For computer software, the software product can be stored in a computer-readable storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or distributed across a network, as long as it enables electronic devices to execute the methods according to the present invention.

[0104] Example 1

[0105] A smart radiotherapy planning method, taking cervical cancer as an example, has the following specific implementation steps:

[0106] 1) Collect historical patient tumor precision radiotherapy planning data and divide it into training set and validation set. The training set includes CT data and delineation data. Through the preprocessing module, the input data is obtained with a fixed size of 1×10×128×128×128, where the second dimension data represents the number of channels, i.e. the number of input features.

[0107] 2) Use the pre-trained MB-UNet model to predict the three-dimensional dose distribution of the training data, with a size of 1×128×128×128.

[0108] 3) Calculate the DVH from the predicted 3D dose distribution, and extract key ROIs and their corresponding dose-volume constraints (30 in total) from the DVH curve: such as target region D. 95 >4500cGy, bladder V 45 <45% (requires conversion) For each constraint, set limit adjustment actions: increase by 2%, keep unchanged, and decrease by 2%. For each constraint, set weight adjustment actions: increase by 10%, keep unchanged, and decrease by 10%, to obtain a discrete action set with 180 elements.

[0109] 4) Construct the value function q using an embedded MB-UNet model. * The estimation model Q(s,a;W) for (s,a) is as follows: Figure 1As shown, s represents the state, a represents the action, and W represents the model parameters, including: a) input, which is the dose distribution. The initial input is the predicted three-dimensional dose distribution, and the subsequent input is the planned optimized three-dimensional dose distribution, which is resampled back to 128×128×128; b) embedding layer, represented by the MB-UNet model, where the input size is changed to 1×128×128×128 and the output size is changed to 1×1283; c) linear layer, where three linear layers are stacked to obtain the output, which is 1×the size of the discrete action set, representing the Q value of each action, and the action with the largest Q value is selected as the current action.

[0110] 5) Define the scoring function using a piecewise function. For each key metric, the score is defined as between 0 and 1, such as... Figure 2 As shown, the segmented thresholds are provided by a senior physicist, and the total score is calculated to obtain the reward function.

[0111] 6) The model is trained by combining empirical replay and a fixed Q-objective to define the loss function.

[0112]

[0113] Where γ is the discount factor, and during the training of the deep Q-network model, a number of samples are randomly drawn from the replay buffer for training. Simultaneously, the deep Q-network model's... The network parameters are kept relatively constant during training and updated to the latest parameters every 20 epochs. During training, each patient's data is set to a maximum of 100 iterations, and 100 iterations are performed sequentially for each training data set. This is the initialization scenario s. 0 For the three-dimensional dose distribution predicted by the dose prediction model, action a is selected using the ∈-greedy strategy. 0 Using a 0 Update the constraints and then call the TPS optimization interface to obtain the dose distribution, i.e., the next scene s. 1 Calculate the reward function r 0 Finally, the sample (s) is stored in the playback buffer. 0 ,a 0 ,r 0 ,s 1 Finally, the parameters of the deep Q-network model are trained using the experience replay strategy.

[0114] 7) After training the deep Q-network model, for a new patient dataset, first predict the three-dimensional dose distribution using the dose prediction model, then use the deep Q-network model to select action a. 0 Thus, the reward function r is obtained. 0 Continue, and obtain the reward function r. 1 ,r2 …r n When n reaches the set value of 5, the optimization ends, and the final plan is obtained. The automatic plan (left) is compared with the original plan (right). Figure 3 As shown.

[0115] Example 2

[0116] See Figure 5 As shown, the dose prediction model in this embodiment is built based on the UNet model and is obtained by training the UNet model by inputting the training dataset into it, including:

[0117] The encoding part consists of a mixing module and a downsampling layer, which is used to extract data features. The downsampling layer uses max-point;

[0118] The decoding part consists of a decoding layer mixing module and an upsampling layer, which is used to gradually restore the original resolution size. The upsampling layer adopts transposed convolution.

[0119] The skip connection part is used to connect the encoding part and the decoding part, and extracts features from the encoding part that help the decoding part restore the original resolution;

[0120] The hybrid module consists of multiple branch paths, and the outputs of the multiple branch paths are added together to obtain the final output.

[0121] The dose prediction model in this embodiment is a hybrid UNet model built on the UNet model. The encoding and decoding parts are respectively introduced into a hybrid module. The hybrid module has multiple branch paths. In this way, when using the deep learning model to predict the radiotherapy dose distribution, the multiple branch paths of the hybrid module can perform different convolution operations on the input data to obtain different representations. Then, the convolution results of each branch are added together and output to achieve fast and accurate radiotherapy dose distribution prediction and generate a radiotherapy plan.

[0122] See Figure 6 As shown, in the deep learning model for predicting radiotherapy dose distribution in this embodiment, the hybrid module includes:

[0123] The first branch path uses convolutional layer L1 to increase dimensionality;

[0124] The second branch path convolutional network L2 includes a first convolutional layer for increasing the channel dimension, a second convolutional layer for depth separation, and a third convolutional layer for reducing the channel dimension and mixing the information of each channel. The number of convolutional kernels in the second convolutional layer is greater than the number of convolutional kernels in the first and third convolutional layers.

[0125] The third branch path convolutional layer L3 is a dense convolutional network, DenseNet, used for feature reuse.

[0126] The first branch path convolutional layer L1, the second branch path convolutional network L2, and the third branch path convolutional layer L3 are configured in parallel. The input data is fed into the first branch path convolutional layer L1, the second branch path convolutional network L2, and the third branch path convolutional layer L3 for convolution processing, respectively. The outputs of the first branch path convolutional layer L1, the second branch path convolutional network L2, and the third branch path convolutional layer L3 are added together as the final output.

[0127] See Figure 6 In the example shown, the leftmost first branch path convolutional layer L1 in this embodiment is a 1×1×1 convolutional layer, which is used to increase dimensionality. The middle second branch path convolutional network L2 is a grouped convolutional network with an inverse bottleneck layer. It starts with a 1×1×1 first convolutional layer to increase channel dimension, followed by a 3×3×3 second convolutional layer, and finally a 1×1×1 third convolutional layer to reduce channel dimension and mix channel information. The rightmost convolutional network is DenseNet, which is used for feature reuse. The outputs of the three branches are added together to obtain the final output. Furthermore, the loss function of the dose prediction model in this embodiment combines a loss function for dose and a loss function for DVH.

[0128] The loss function for dose measures the average absolute error between the predicted dose and the actual dose of pixels within the contour. No prediction is made for pixels outside the contour, since the actual dose outside the contour is 0. This also prevents an excessive number of 0 doses from causing the overall predicted dose to be too low.

[0129] The loss function for DVH described in this embodiment includes dose loss at the percentile position and first derivative loss. The loss function for DVH is:

[0130] dvh loss(pred,gt)=L1(pred[sort index],, sort(gt))+L1(pred[sort index], sort′(gt))

[0131] Where pred represents the predicted dose, gt represents the actual dose, L1 represents the mean absolute loss, and sort represents reordering the actual dose and using the original indices of the sorted elements for sorting the predicted dose.

[0132] As an optional implementation of this embodiment, the training method of the dose prediction model in this embodiment includes:

[0133] Collect historical data on precision radiotherapy for tumors in patients, including CT data, organ delineation data, planned prescription data, and dose data;

[0134] The collected planning data is preprocessed to make it suitable for reading by the UNet model;

[0135] Data augmentation is performed on the preprocessed planning data to generate more data and obtain the training dataset.

[0136] The UNet model is trained using the training dataset to obtain a deep learning model.

[0137] The preprocessing of the collected planning data described in this embodiment includes:

[0138] Resample, match, and crop CT data, dose data, and organ delineation data to a preset size;

[0139] The CT data is scaled as follows: regions with HU values ​​less than the first preset HU value are set to the first preset HU value, regions with HU values ​​greater than the second preset HU value are set to the second preset HU value, and regions with HU values ​​between the first preset HU value and the second preset HU value are set to the original HU value. The output is in a format for UNet model reading.

[0140] The data augmentation process for the preprocessed planning data described in this embodiment includes:

[0141] Simultaneously perform data enhancement operations on preprocessed CT data, dose data, and organ delineation data;

[0142] The data augmentation operations include translation, flipping, and rotation. Translation sets a maximum offset, and the translated data is restored to its original size by padding with zeros. Flipping randomly flips the three axes, and rotation randomly selects a rotation angle from fixed angle values.

[0143] This embodiment also provides a method for generating a radiotherapy plan, including:

[0144] The user's CT data and organ delineation data are acquired and input into the deep learning model for predicting the radiotherapy dose distribution.

[0145] The deep learning model is used to predict and output a three-dimensional dose distribution.

[0146] This embodiment of a radiotherapy plan generation method inputs the user's CT data and organ delineation data into the dose prediction model. The dose prediction model is a hybrid UNet model built based on the UNet model. The encoding and decoding parts are respectively introduced into a hybrid module. The hybrid module has multiple branch paths. In this way, when using the dose prediction model to predict the radiotherapy dose distribution, the multiple branch paths of the hybrid module can perform different convolution operations on the input data to obtain different representations. Then, the convolution results of each branch are added together and output to achieve fast and accurate radiotherapy dose distribution prediction and generate a radiotherapy plan.

[0147] Furthermore, the radiotherapy plan generation method of this embodiment guides optimization based on the predicted three-dimensional dose distribution to generate the final plan; specifically, it includes: calculating the dose-dependent volume (DVH) of each organ based on the three-dimensional dose distribution output by the dose prediction model, setting DVH constraints based on the DVH of each organ, and using an optimization algorithm to optimize the three-dimensional dose distribution to obtain the radiotherapy plan.

[0148] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described herein. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the specific embodiments described above. Therefore, any modifications or equivalent substitutions to the present invention, as well as all technical solutions and improvements that do not depart from the spirit and scope of the invention, are covered within the scope of the claims of the present invention.

Claims

1. A method of intelligent planning of radiotherapy, characterized in that, The method comprises the following steps: inputting radiotherapy plan input information of a patient, and predicting a three-dimensional dose distribution of the patient through a dose prediction model; generating an initial radiotherapy plan based on the radiotherapy plan input information and the three-dimensional dose distribution of the patient; automatically learning and optimizing parameters of the initial radiotherapy plan through a deep Q network model to obtain a final radiotherapy plan meeting clinical requirements; the automatic learning and optimization of the parameters of the initial radiotherapy plan through the deep Q network model comprises: obtaining a DVH curve from the three-dimensional dose distribution, and extracting key ROIs and corresponding dose volume constraints from the DVH curve; setting adjustment actions for the dose volume constraints of each key ROI: fixed proportion up-regulation of the limit value, fixed proportion down-regulation of the limit value, fixed proportion up-regulation of the weight, fixed proportion down-regulation of the weight, and fixed proportion up-regulation of the weight, and obtaining a discrete action set; the deep Q network model comprises: Adopting an embedded dose prediction model to construct a value function of an estimation model where s represents a state, a represents an action, and W represents a model parameter Defining the score function For each key ROI, the score is defined as a continuous function between 0-1, the score sum is counted, and then the reward function is obtained ; defining a loss function in combination with experience replay and fixed Q target: wherein, is a discount factor, and the experience replay specifically does the following: maintain a replay buffer, store the quadruple of state, action, reward, next state sampled from the environment each time, and when training the deep Q network model, randomly draw a number of samples from the replay buffer to train the deep Q network model, and the , and update the network parameters to the latest network parameters every certain training period: ; the dose prediction model is constructed based on a UNet model, is trained by inputting a training data set into the UNet model, and comprises: an encoding part composed of a hybrid module and a down-sampling layer, which is used for extracting data features, and the down-sampling layer uses maxpool; a decoding part composed of a decoding layer hybrid module and an up-sampling layer, which is used for gradually restoring the original resolution size, and the up-sampling layer adopts transpose convolution; a skip connection part used for connecting the encoding part and the decoding part, and extracting features from the encoding part that are helpful for the decoding part to restore to the original resolution; wherein the hybrid module is composed of a plurality of branch paths, and outputs of the plurality of branch paths are added as a final output; the hybrid module comprises: a first branch path convolution layer L1 used for improving the dimension; a second branch path convolution network L2 comprising a first convolution layer used for improving the channel dimension, a second convolution layer used for depth separation, and a third convolution layer used for reducing the channel dimension and mixing information of each channel, and the number of convolution kernels of the second convolution layer is greater than that of the first convolution layer and the third convolution layer; a third branch path convolution layer L3 being a DenseNet used for feature reuse; the first branch path convolution layer L1, the second branch path convolution network L2, and the third branch path convolution layer L3 are arranged side by side, input data enter the first branch path convolution layer L1, the second branch path convolution network L2, and the third branch path convolution layer L3 for convolution processing respectively, and output results of the first branch path convolution layer L1, the second branch path convolution network L2, and the third branch path convolution layer L3 are added as a final output.

2. The radiotherapy intelligent planning method according to claim 1, wherein: The estimation model comprises: a) input, the input is a dose distribution, the initial input is a three-dimensional dose distribution predicted by a dose prediction model, and then the input is a three-dimensional dose distribution of plan optimization; b) an Embedding layer represented by a dose prediction model; c) a linear layer, several linear layers are stacked to obtain an output, and the output is: 1´discrete action set size, representing the Q value of each action, and the action with the maximum Q value is selected as the current action.

3. The intelligent radiotherapy planning method of claim 1, wherein, The automatic learning parameter optimization for the initial radiotherapy plan through the deep Q network model, wherein the training process of the deep Q network model comprises: Collecting the planning data of precise radiotherapy of historical patients, and dividing the planning data into a training set and a validation set, wherein the training set comprises CT data and key ROI delineation data, and the validation set comprises a final radiotherapy plan; Using the trained dose prediction model to predict the training set data to obtain a three-dimensional dose distribution; Obtaining a DVH curve through the three-dimensional dose distribution, extracting key ROIs and corresponding dose volume constraints from the DVH curve, setting adjustment actions for the dose volume constraints of each key ROI, and obtaining a discrete action set; Adopting an embedded dose prediction model to construct a value function An estimation model An estimation model The initial input of the estimation model is the three-dimensional dose distribution predicted by the dose prediction model, and the subsequent input is the three-dimensional dose distribution optimized by the plan, and the output is: 1 x the size of the discrete action set, indicating the Q value of each action, and the action with the maximum Q value is selected as the current action Score function is defined using piecewise function For each key ROI, score is defined between 0-1, piecewise threshold is set by senior physicist, score sum is counted, and reward function is obtained ; The model is trained by using a loss function defined by combining experience replay and a fixed Q-target, wherein, is a discount factor, and a number of samples are randomly drawn from the replay buffer for training the deep Q-network model, and the During the training process, the target network is kept relatively fixed, and is updated with the latest network parameters every preset number of rounds: ; During the training of the deep Q-network model, a maximum number of iterations is set for each patient's data set, and the maximum number of iterations is completed sequentially for each training set data set: that is, initializing the scenario. For the three-dimensional dose distribution predicted by the dose prediction model, use Strategy Selection Action ,use The constraints are updated, and then the radiation therapy planning system optimization interface is invoked to obtain the dose distribution, i.e., the next scenario. Calculate the reward function Finally, the samples are stored in the playback buffer. Finally, the parameters of the deep Q-network model are trained using the experience replay strategy.

4. The intelligent radiotherapy planning method of claim 3, wherein, The automatic learning parameter optimization for the initial radiotherapy plan through the deep Q network model, wherein the training process of the deep Q network model comprises: When the deep Q network model is trained, a new patient data is inputted; Predicting the three-dimensional dose distribution by the dose prediction model, selecting the action using the trained deep Q network model , and obtaining the reward function , continuing, obtaining the reward function , ending the optimization when n reaches the set value, and obtaining the final plan.

5. The intelligent radiotherapy planning method of any one of claims 1-4, wherein, The radiotherapy plan input information of the patient comprises patient CT, target region and key ROI delineation, and the radiotherapy plan input information is standardized; After the radiotherapy plan input information is standardized, all the radiotherapy plan input information, the distance features extracted from the radiotherapy plan input information and the beam mask features extracted from the target region are converted into a fixed size, and then combined into the input of the dose prediction model.

6. A radiotherapy planning system implementing the method of intelligent radiotherapy planning according to any one of claims 1 to 5, characterized in that, Comprise: A pre-processing and input module for inputting the radiotherapy plan input information of the patient; A dose prediction module for predicting the three-dimensional dose distribution of the patient through the dose prediction model, and generating an initial radiotherapy plan based on the radiotherapy plan input information of the patient and the three-dimensional dose distribution; An automatic parameter optimization module for automatically learning and optimizing the parameters of the initial radiotherapy plan through the deep Q network model to obtain a final radiotherapy plan meeting the clinical requirements.

Images