Radiotherapy intelligent planning method and system

Abstract

The invention discloses a radiotherapy intelligent planning method and system, and the method comprises the steps: inputting the radiotherapy plan input information of a patient, and predicting the three-dimensional dose distribution of the patient through a dose prediction model; generating an initial radiotherapy plan based on radiotherapy plan input information of the patient and the three-dimensional dose distribution; and performing automatic learning parameter adjustment and optimization on the initial radiotherapy plan through the deep Q network model to obtain a final radiotherapy plan meeting clinical requirements. According to the radiotherapy intelligent planning method, automatic prediction of three-dimensional dose distribution is realized through the dose prediction model, automatic parameter optimization is carried out on the initial radiotherapy plan of the three-dimensional dose distribution through the depth Q network model, and finally, the final radiotherapy plan meeting clinical requirements is automatically generated, so that the radiotherapy efficiency is improved. According to the radiotherapy intelligent planning method, a model combining dose prediction and automatic parameter optimization is established, automatic, accurate and efficient radiotherapy plan optimization is realized, and the method has important practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of radiotherapy planning, and in particular to a radiotherapy intelligent planning method and system. Background Art

[0002] In the radiotherapy industry, a radiotherapy plan is a document that controls the accelerator to project radiation. Just like the human brain controls human behavior, how the accelerator treats the patient is controlled by the radiotherapy plan. The radiotherapy plan determines the patient's efficacy and the severity of the late side effects. During the entire treatment period, once the radiotherapy plan is designed, it is rarely modified unless the patient's irradiation range needs to be modified midway. Therefore, the radiotherapy plan is very important.

[0003] In the existing radiotherapy planning system, the parameters of the target area and organs at risk are mainly adjusted manually, and the degree of intelligence and automation is low. Therefore, the existing application number is 202211144237.8, and the name is an automatic planning method and system based on dose prediction and parameter optimization. The method includes: performing a planned CT scan on the patient to obtain a CT image, and obtaining organ delineation data based on the CT image; inputting the CT image and organ delineation data into the trained 3D VGG-U-Net network model to obtain a three-dimensional dose distribution prediction of the organs at risk, and obtaining the average dose of the organs at risk based on the three-dimensional dose distribution prediction; determining the objective function of the irradiation field distribution model based on the organ delineation data and the average dose of the organs at risk to design an initial radiotherapy plan, and optimizing the parameters of the objective function based on the particle swarm and genetic hybrid algorithm to obtain the parameter optimization result; evaluating the parameter optimization result, updating the parameters of the objective function according to the evaluation result, and solving the objective function according to the parameter update result to obtain the best radiotherapy plan. The invention realizes the function of automatically completing the parameter optimization in the radiotherapy plan by the intelligent algorithm, which is convenient for improving the quality and efficiency of the plan.

[0004] The above-mentioned invention patent optimizes the objective function by using a particle swarm and genetic hybrid optimization algorithm after dose prediction. However, the invention patent still has technical defects in that the algorithm calculation complexity is too high and only the average dose information of the organs at risk is used in predicting the three-dimensional dose distribution.

[0005] In view of this, the present invention patent is proposed. Summary of the invention

[0006] In order to solve the above problems, the present invention provides a radiotherapy intelligent planning method and system, which specifically adopts the following technical solutions:

[0007] A radiotherapy intelligent planning method, comprising:

[0008] Input the patient's radiotherapy plan input information and predict the patient's three-dimensional dose distribution through the dose prediction model;

[0009] Generate an initial radiotherapy plan based on the patient's radiotherapy plan input information and three-dimensional dose distribution;

[0010] The deep Q network model is used to automatically learn and tune the parameters of the initial radiotherapy plan to obtain the final radiotherapy plan that meets clinical requirements.

[0011] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the automatic learning parameter tuning for the initial radiotherapy plan by using a deep Q network model includes:

[0012] The DVH curve is obtained through three-dimensional dose distribution, and the key ROI and its corresponding dose volume constraint are extracted from the DVH curve;

[0013] Adjustment actions are set for the dose volume constraints of each key ROI: limit value is increased by a fixed ratio, limit value remains unchanged, limit value is decreased by a fixed ratio, and weight is increased by a fixed ratio, weight remains unchanged, and weight is decreased by a fixed ratio to obtain a discrete action set.

[0014] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the automatic learning parameter tuning is performed for the initial radiotherapy plan by using a deep Q network model, wherein the deep Q network model includes:

[0015] Using the embedded dose prediction model to construct the value function q * (s,a) estimation model Q(s,a;W), where s represents the state, a represents the action, and W represents the model parameters;

[0016] The estimation model Q(s, a; W) includes:

[0017] a) Input: The input is dose distribution. The initial input is the three-dimensional dose distribution predicted by the dose prediction model, and the subsequent input is the three-dimensional dose distribution of the plan optimization.

[0018] b) Embedding layer, represented by the dose prediction model;

[0019] c) Linear layer, stack several layers of linear layers to get the output, the output is: 1×discrete action set size, representing the Q value of each action, and select the action with the maximum Q value as the current action.

[0020] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the automatic learning parameter tuning is performed for the initial radiotherapy plan by using a deep Q network model, wherein the deep Q network model includes:

[0021] Define the score function For each key ROI, the score is defined as a continuous function between 0 and 1, and the sum of the scores is calculated to obtain the reward function

[0022] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the automatic learning parameter tuning is performed for the initial radiotherapy plan by using a deep Q network model, wherein the deep Q network model includes:

[0023] The loss function is defined by combining experience replay and fixed Q target:

[0024]

[0025] Among them, γ is the discount factor. The specific practice of experience replay is to maintain a replay buffer, store the four-element array (state, action, reward, next state) sampled from the environment each time into the replay buffer, and randomly extract several samples from the replay buffer for training when training the deep Q network model. Keep it relatively fixed during the training process and update it to the latest network parameters every certain training cycle:

[0026] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the automatic learning parameter tuning is performed on the initial radiotherapy plan by using a deep Q network model, wherein the training process of the deep Q network model includes:

[0027] Collect historical patient tumor precision radiotherapy planning data and divide it into training set and validation set. The training set includes CT data and key ROI delineation data, and the validation set includes the final radiotherapy plan.

[0028] Use the trained dose prediction model to predict the training set data to obtain the three-dimensional dose distribution;

[0029] The DVH curve is obtained through the three-dimensional dose distribution, and the key ROI and its corresponding dose volume constraint are extracted from the DVH curve. The adjustment action is set for the dose volume constraint of each key ROI to obtain a discrete action set;

[0030] Using the embedded dose prediction model to construct the value function q * The estimation model Q(s,a;W) of (s,a) is initially inputted as the three-dimensional dose distribution predicted by the MB-UNet model, and then as the three-dimensional dose distribution of the planned optimization. The output is: 1×discrete action set size, which represents the Q value of each action. The action with the maximum Q value is selected as the current action.

[0031] Define the score function using piecewise function For each key ROI, the score is defined between 0 and 1. The segmentation threshold is set by a senior physicist. The total score is calculated to obtain the reward function.

[0032] The model is trained by combining experience replay and fixed Q target to define the loss function. Among them, γ is the discount factor. When training the deep Q network model, several samples are randomly extracted from the playback buffer for training. At the same time, the deep Q network model During the training process, the parameters are kept relatively fixed and updated to the latest network parameters every preset number of rounds:

[0033] When training the deep Q network model, the maximum number of iterations is set for each patient data, and the maximum number of iterations of each training set data is completed in turn: that is, the initialization scene s 0 For the three-dimensional dose distribution predicted by the dose prediction model, the ∈-greedy strategy is used to select action a 0 , using a 0 Update the constraints and then call the radiotherapy planning system optimization interface to obtain the dose distribution, that is, the next scenario s 1 , calculate the reward function r 0 , and finally store the sample (s) in the playback buffer 0 ,a 0 ,r 0 ,s 1 ), and finally the deep Q network model parameters are trained by the experience replay strategy.

[0034] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the automatic learning parameter tuning of the initial radiotherapy plan by the deep Q network model to obtain the final radiotherapy plan that meets clinical requirements includes:

[0035] When the deep Q network model is trained, a new patient data is input;

[0036] The three-dimensional dose distribution is predicted by the dose prediction model, and the action a is selected using the trained deep Q network model. 0 , and then obtain the reward function r 0 , continue and get the reward function r 1 ,r 2 …r n , when n reaches the set value, the optimization ends and the final plan is obtained.

[0037] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the radiotherapy plan input information input to the patient includes: patient CT, target area and key ROI delineation, and the radiotherapy plan input information is standardized;

[0038] After the radiotherapy plan input information is standardized, all radiotherapy plan input information, distance features extracted from the radiotherapy plan input information, and beam mask features extracted from the target area are converted into fixed sizes and then combined into inputs of the dose prediction model.

[0039] As an optional embodiment of the present invention, in a radiotherapy intelligent planning method of the present invention, the dose prediction model uses a multi-branch UNet model (MB-UNet), and the multi-branch UNet model is a "U"-shaped structure, including a coding layer, a decoding layer and a skip connection layer.

[0040] The present invention also provides a radiotherapy planning system for implementing the radiotherapy intelligent planning method, comprising:

[0041] The pre-processing and input module inputs the patient’s radiotherapy plan input information;

[0042] The dose prediction module predicts the patient's three-dimensional dose distribution through the dose prediction model, and generates an initial radiotherapy plan based on the patient's radiotherapy plan input information and the three-dimensional dose distribution;

[0043] The automatic parameter optimization module uses a deep Q network model to automatically learn and tune the parameters of the initial radiotherapy plan to obtain the final radiotherapy plan that meets clinical requirements.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The intelligent radiotherapy planning method of the present invention realizes automatic prediction of three-dimensional dose distribution through a dose prediction model, and at the same time, automatically optimizes the parameters of the initial radiotherapy plan of the three-dimensional dose distribution through a deep Q network model, and finally automatically generates a final radiotherapy plan that meets clinical requirements. Therefore, the intelligent radiotherapy planning method of the present invention establishes a model that combines dose prediction and automatic parameter optimization, realizes automatic, accurate and efficient optimization of radiotherapy plans, and has important practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 In the first embodiment of the present invention, the embedded MB-UNet model is used to construct the value function q * Framework diagram of the estimation model Q(s,a;W) of (s,a);

[0047] Figure 2An example diagram of the score distribution of each key ROI in the first embodiment of the present invention;

[0048] Figure 3 A schematic diagram showing a comparison between the automatic plan and the original plan in the first embodiment of the present invention;

[0049] Figure 4 A module diagram of a radiotherapy intelligent planning system according to an embodiment of the present invention;

[0050] Figure 5 Principle framework diagram of the dose prediction model in the second embodiment of the present invention;

[0051] Figure 6 A specific example diagram of the mixing module in the dose prediction model in the second embodiment of the present invention. DETAILED DESCRIPTION

[0052] To make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be described clearly and completely in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them.

[0053] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the invention claimed for protection, but merely represents some embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0054] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions in the embodiments may be combined with each other.

[0055] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0056] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the invention product is usually placed when in use, or the orientation or positional relationship commonly understood by those skilled in the art. Such terms are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0057] See also Figure 1As shown, a radiotherapy intelligent planning method of this embodiment includes:

[0058] Input the patient's radiotherapy plan input information and predict the patient's three-dimensional dose distribution through the dose prediction model;

[0059] Generate an initial radiotherapy plan based on the patient's radiotherapy plan input information and three-dimensional dose distribution;

[0060] The deep Q network model is used to automatically learn and tune the parameters of the initial radiotherapy plan to obtain the final radiotherapy plan that meets clinical requirements.

[0061] The intelligent radiotherapy planning method of this embodiment realizes automatic prediction of three-dimensional dose distribution through a dose prediction model, and at the same time, automatically optimizes the parameters of the initial radiotherapy plan of the three-dimensional dose distribution through a deep Q network model, and finally automatically generates a final radiotherapy plan that meets clinical requirements. Therefore, the intelligent radiotherapy planning method of this embodiment establishes a model that combines dose prediction and automatic parameter optimization, realizes automatic, accurate and efficient optimization of radiotherapy plans, and has important practical value.

[0062] As an optional implementation of this embodiment, in a radiotherapy intelligent planning method of this embodiment, the automatic learning parameter tuning for the initial radiotherapy plan by using a deep Q network model includes:

[0063] The DVH curve is obtained through three-dimensional dose distribution, and the key ROI and its corresponding dose volume constraint are extracted from the DVH curve;

[0064] Adjustment actions are set for the dose volume constraints of each key ROI: limit value is increased by a fixed ratio, limit value remains unchanged, limit value is decreased by a fixed ratio, and weight is increased by a fixed ratio, weight remains unchanged, and weight is decreased by a fixed ratio to obtain a discrete action set.

[0065] Furthermore, in this embodiment, the deep Q network model is used to automatically learn and tune parameters for the initial radiotherapy plan, wherein the deep Q network model includes:

[0066] Using the embedded dose prediction model to construct the value function q * (s,a) estimation model Q(s,a;W), where s represents the state, a represents the action, and W represents the model parameters;

[0067] The estimation model Q(s, a; W) includes:

[0068] a) Input: The input is dose distribution. The initial input is the three-dimensional dose distribution predicted by the dose prediction model, and the subsequent input is the three-dimensional dose distribution of the plan optimization.

[0069] b) Embedding layer, represented by the dose prediction model;

[0070] c) Linear layer, stack several layers of linear layers to get the output, the output is: 1×discrete action set size, representing the Q value of each action, and select the action with the maximum Q value as the current action.

[0071] Furthermore, in this embodiment, the deep Q network model is used to automatically learn and tune parameters for the initial radiotherapy plan, wherein the deep Q network model includes:

[0072] Define the score function For each key ROI, the score is defined as a continuous function between 0 and 1, and the sum of the scores is calculated to obtain the reward function

[0073] Furthermore, in this embodiment, the deep Q network model is used to automatically learn and tune parameters for the initial radiotherapy plan, wherein the deep Q network model includes:

[0074] The loss function is defined by combining experience replay and fixed Q target:

[0075]

[0076] Among them, γ is the discount factor. The specific practice of experience replay is to maintain a replay buffer, store the four-element array (state, action, reward, next state) sampled from the environment each time into the replay buffer, and randomly extract several samples from the replay buffer for training when training the deep Q network model. Keep it relatively fixed during the training process and update it to the latest network parameters every certain training cycle:

[0077] As an optional implementation of this embodiment, the automatic learning parameter tuning for the initial radiotherapy plan is performed by a deep Q network model in this embodiment, wherein the training process of the deep Q network model includes:

[0078] Collect historical patient tumor precision radiotherapy planning data and divide it into training set and validation set. The training set includes CT data and key ROI delineation data, and the validation set includes the final radiotherapy plan.

[0079] Use the trained dose prediction model to predict the training set data to obtain the three-dimensional dose distribution;

[0080] The DVH curve is obtained through the three-dimensional dose distribution, and the key ROI and its corresponding dose volume constraint are extracted from the DVH curve. The adjustment action is set for the dose volume constraint of each key ROI to obtain a discrete action set;

[0081] Using the embedded dose prediction model to construct the value function q * The estimation model Q(s,a;W) of (s,a) is initially inputted as the three-dimensional dose distribution predicted by the MB-UNet model, and then as the three-dimensional dose distribution of the planned optimization. The output is: 1×discrete action set size, which represents the Q value of each action. The action with the maximum Q value is selected as the current action.

[0082] Define the score function using piecewise function For each key ROI, the score is defined between 0 and 1. The segmentation threshold is set by a senior physicist. The total score is calculated to obtain the reward function.

[0083] The model is trained by combining experience replay and fixed Q target to define the loss function. Among them, γ is the discount factor. When training the deep Q network model, several samples are randomly extracted from the playback buffer for training. At the same time, the deep Q network model During the training process, the parameters are kept relatively fixed and updated to the latest network parameters every preset number of rounds:

[0084] When training the deep Q network model, the maximum number of iterations is set for each patient data, and the maximum number of iterations of each training set data is completed in turn: that is, the initialization scene s 0 For the three-dimensional dose distribution predicted by the dose prediction model, the ∈-greedy strategy is used to select action a 0 , using a 0 Update the constraints and then call the radiotherapy planning system optimization interface to obtain the dose distribution, that is, the next scenario s 1 , calculate the reward function r 0 , and finally store the sample (s) in the playback buffer 0 ,a 0 ,r 0 ,s 1 ), and finally the deep Q network model parameters are trained by the experience replay strategy.

[0085] In a radiotherapy intelligent planning method of this embodiment, the automatic learning parameter tuning of the initial radiotherapy plan by the deep Q network model to obtain a final radiotherapy plan that meets clinical requirements includes:

[0086] When the deep Q network model is trained, a new patient data is input;

[0087] The three-dimensional dose distribution is predicted by the dose prediction model, and the action a is selected using the trained deep Q network model. 0, and then obtain the reward function r 0 , continue and get the reward function r 1 ,r 2 …r n , when n reaches the set value, the optimization ends and the final plan is obtained.

[0088] In a radiotherapy intelligent planning method of this embodiment, the radiotherapy plan input information inputted to the patient includes: patient CT, target area and key ROI delineation, and the radiotherapy plan input information is standardized;

[0089] After the radiotherapy plan input information is standardized, all radiotherapy plan input information, distance features extracted from the radiotherapy plan input information, and beam mask features extracted from the target area are converted into fixed sizes and then combined into inputs of the dose prediction model.

[0090] The standardization process includes: resampling, dose alignment, cropping and patching, shrinking and expanding, and calculating a distance matrix.

[0091] In a radiotherapy intelligent planning method of this embodiment, the dose prediction model uses a multi-branch UNet model (MB-UNet), and the multi-branch UNet model is a "U"-shaped structure, including a coding layer, a decoding layer and a skip connection layer.

[0092] See also Figure 4 As shown, this embodiment also provides a radiotherapy planning system for implementing the radiotherapy intelligent planning method, including:

[0093] The pre-processing and input module inputs the patient’s radiotherapy plan input information;

[0094] The dose prediction module predicts the patient's three-dimensional dose distribution through the dose prediction model, and generates an initial radiotherapy plan based on the patient's radiotherapy plan input information and the three-dimensional dose distribution;

[0095] The automatic parameter optimization module uses a deep Q network model to automatically learn and tune the parameters of the initial radiotherapy plan to obtain the final radiotherapy plan that meets clinical requirements.

[0096] This embodiment also provides a computer-readable storage medium storing a computer-executable program. When the computer-executable program is executed, the radiotherapy intelligent planning method as described above is implemented.

[0097] The computer-readable storage medium described in this embodiment may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus, or a device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0098] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory is used to store a computer executable program, and when the computer program is executed by the processor, the processor executes the radiotherapy intelligent planning method.

[0099] The electronic device is presented in the form of a general-purpose computing device. The processor may be one or more than one and work in coordination. The present invention does not exclude distributed processing, that is, the processor may be dispersed in different physical devices. The electronic device of the present invention is not limited to a single entity, but may also be the sum of multiple physical devices.

[0100] The memory stores a computer executable program, which is usually a machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least part of the steps in the method.

[0101] The memory includes a volatile memory, such as a random access memory unit (RAM) and / or a cache memory unit, and may also be a non-volatile memory, such as a read-only memory unit (ROM).

[0102] It should be understood that the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as display screens, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. As long as the electronic device can execute the computer-readable program in the memory to implement the method of the present invention or at least part of the steps of the method, it can be considered as an electronic device covered by the present invention.

[0103] Through the above description of the implementation mode, it is easy for those skilled in the art to understand that the present invention can be implemented by hardware capable of executing a specific computer program, such as the system of the present invention, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. contained in the system. The present invention can also be implemented by computer software that executes the method of the present invention, such as control software executed by a microprocessor, an electronic control unit, a client, a server, etc. However, it should be noted that the computer software that executes the method of the present invention is not limited to being executed by one or a specific hardware entity, and it can also be implemented in a distributed manner by unspecified specific hardware. For computer software, the software product can be stored in a computer-readable storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), and can also be distributed and stored on the network, as long as it can enable the electronic device to execute the method according to the present invention.

[0104] Embodiment 1

[0105] A radiotherapy intelligent planning method, taking cervical cancer as an example, the specific implementation steps are as follows:

[0106] 1) The planning data of historical patient tumor precision radiotherapy are collected and divided into training set and validation set. The training set includes CT data and contouring data. Through the preprocessing module, the input data is obtained with a fixed size of 1×10×128×128×128, where the second dimension data represents the number of channels, that is, the number of input features.

[0107] 2) Use the trained MB-UNet model to predict the training data to obtain a three-dimensional dose distribution with a size of 1×128×128×128.

[0108] 3) Calculate DVH from the predicted 3D dose distribution, and extract key ROIs and their corresponding dose volume constraints (30 in total) from the DVH curve: 95 >4500cGy, bladder V 45 <45% (need to be converted ), etc., set limit adjustment actions for each constraint: increase by 2%, remain unchanged and decrease by 2%, set weight adjustment actions for each constraint: increase by 10%, remain unchanged and decrease by 10%, and obtain a discrete action set with 180 elements.

[0109] 4) Use the embedded MB-UNet model to construct the value function q * The estimation model Q(s,a;W) of (s,a) is as follows: Figure 1As shown in the figure, s represents state, a represents action, and W represents model parameters, including: a) input, the input is dose distribution, the initial input is the predicted three-dimensional dose distribution, and the subsequent input is the planned optimized three-dimensional dose distribution, and it is resampled back to 128×128×128, b) Embedding layer, represented by MB-UNet model, where the input size is changed to 1×128×128×128, and the output size is changed to 1×1283, c) Linear layer, stacking 3 linear layers to get the output, the output is: 1×discrete action set size, representing the Q value of each action, and selecting the action with the maximum Q value as the current action.

[0110] 5) Define the scoring function using piecewise function For each key indicator, a score is defined between 0 and 1, such as Figure 2 As shown, the segmentation threshold is given by a senior physicist, and the total score is calculated to obtain the reward function

[0111] 6) The loss function training model is defined by combining experience replay and fixed Q target.

[0112]

[0113] Among them, γ is the discount factor. When training the deep Q network model, several samples are randomly extracted from the playback buffer for training. At the same time, the deep Q network model Keep it relatively fixed during the training process and update it to the latest network parameters every 20 rounds: During training, the maximum number of iterations for each patient data is set to 100, and 100 rounds of iterations for each training data are completed in sequence. 0 For the three-dimensional dose distribution predicted by the dose prediction model, the ∈-greedy strategy is used to select action a 0 , using a 0 Update the constraints and call the TPS optimization interface to obtain the dose distribution, that is, the next scene s 1 , calculate the reward function r 0 , and finally store the sample (s) in the playback buffer 0 ,a 0 ,r 0 ,s 1 ), and finally the deep Q network model parameters are trained by the experience replay strategy.

[0114] 7) After the deep Q network model is trained, for a new patient data, the three-dimensional dose distribution is first predicted by the dose prediction model, and the deep Q network model is used to select action a 0 , and then obtain the reward function r 0 , continue and get the reward function r 1 ,r2 …r n When n reaches the set value of 5, the optimization ends and the final plan is obtained. The automatic plan (left) is compared with the original plan (right). Figure 3 shown.

[0115] Embodiment 2

[0116] See also Figure 5 As shown, the dose prediction model of this embodiment is constructed based on the UNet model, and is obtained by inputting the training data set into the UNet model for training, including:

[0117] The encoding part, which consists of a mixing module and a downsampling layer, is used to extract data features. The downsampling layer uses maxpool;

[0118] The decoding part is composed of a decoding layer hybrid module and an upsampling layer, which is used to gradually restore the original resolution size. The upsampling layer adopts transposed convolution;

[0119] The skip connection part is used to connect the encoding part and the decoding part, and extract features from the encoding part that help the decoding part to restore to the original resolution;

[0120] Wherein, the mixing module is composed of multiple branch paths, and the outputs of the multiple branch paths are added together as the final output.

[0121] The dose prediction model of this embodiment is a hybrid UNet model constructed based on the UNet model. The encoding part and the decoding part respectively introduce hybrid modules. The hybrid module has multiple branch paths. In this way, when using the deep learning model to predict the radiotherapy dose distribution, the multiple branch paths of the hybrid module can perform different convolution operations on the input data to obtain different representations, and then add the convolution results of each branch to output, so as to achieve fast and accurate radiotherapy dose distribution prediction and generate a radiotherapy plan.

[0122] See also Figure 6 As shown, in the deep learning model for radiotherapy dose distribution prediction of this embodiment, the hybrid module includes:

[0123] The first branch path convolution layer L1 is used to increase the dimension;

[0124] The second branch path convolutional network L2 includes a first convolutional layer for increasing the channel dimension, a second convolutional layer for depth separation, and a third convolutional layer for reducing the channel dimension and mixing the information of each channel, wherein the number of convolutional kernels of the second convolutional layer is greater than the number of convolutional kernels of the first convolutional layer and the third convolutional layer;

[0125] The third branch path convolution layer L3 is a dense convolutional network DenseNet, which is used for feature reuse;

[0126] The first branch path convolution layer L1, the second branch path convolution network L2 and the third branch path convolution layer L3 are configured in parallel, and the input data enters the first branch path convolution layer L1, the second branch path convolution network L2 and the third branch path convolution layer L3 for convolution processing respectively, and the output results of the first branch path convolution layer L1, the second branch path convolution network L2 and the third branch path convolution layer L3 are added as the final output.

[0127] See also Figure 6 In the example shown in , the first branch path convolution layer L1 on the far left of this embodiment is a 1×1×1 convolution layer, which is used to increase the dimension. The second branch path convolution network L2 in the middle is a group convolution network with an inverse bottleneck layer. It starts with a 1×1×1 first convolution layer to increase the channel dimension, followed by a 3×3×3 second convolution layer, and finally a 1×1×1 third convolution layer to reduce the channel dimension and mix the information of each channel. The rightmost is a dense convolution network DenseNet, which is used to reuse features, and the outputs of the three branches are added as the final output. Furthermore, the loss function of the dose prediction model in this embodiment adopts a combination of a loss function for dose and a loss function for dvh.

[0128] Among them, the loss function for dose measures the mean absolute error between the predicted dose and the actual dose of the pixel points within the contour. No prediction is made outside the contour because the actual dose outside the contour is 0, so it is not predicted. At the same time, it can also prevent too much 0 dose from causing the overall predicted dose to be too low.

[0129] The loss function for dvh in this embodiment includes the dose loss at the percentile position and the first-order derivative loss. The loss function for dvh is:

[0130] dvh loss(pred,gt)=L1(pred[sort index],,sort(gt))+L1(pred[sortindice],sort′(gt))

[0131] Among them, pred represents the predicted dose, gt represents the actual dose, L1 represents the mean absolute loss, sort represents the reordering of the actual doses, and the original subscripts corresponding to the sorted elements are used for the predicted dose sorting.

[0132] As an optional implementation of this embodiment, the training method of the dose prediction model of this embodiment includes:

[0133] Collect historical patient tumor precision radiotherapy planning data, including CT data, organ delineation data, planned prescription data, and dose data;

[0134] Preprocess the collected planning data to make it suitable for reading by the UNet model;

[0135] Perform data enhancement processing on the preprocessed planned data to generate more data and obtain a training data set;

[0136] The training data set is used to train the UNet model to obtain a deep learning model.

[0137] The preprocessing of the collected planning data in this embodiment includes:

[0138] Resample, match and crop CT data, dose data and organ delineation data to preset sizes;

[0139] The CT data is scaled: the areas with HU values ​​less than the first preset HU value are set to the first preset HU value, the areas with HU values ​​greater than the second preset HU value are set to the second preset HU value, and the areas with HU values ​​between the first preset HU value and the second preset HU value are set to the original HU value, and the output is in a format for reading by the UNet model.

[0140] The data enhancement processing for the preprocessed planning data in this embodiment includes:

[0141] Data enhancement operations are performed synchronously on the pre-processed CT data, dose data, and organ delineation data;

[0142] The data enhancement operations include translation, flipping and rotation, wherein translation sets a maximum offset, the translated data is filled with 0 to restore the original data size, flipping is a random flipping of three axes, and rotation is a random selection of a rotation angle from a fixed angle value.

[0143] This embodiment also provides a method for generating a radiotherapy plan, including:

[0144] Obtaining the user's CT data and organ delineation data, and inputting them into the deep learning model for radiotherapy dose distribution prediction;

[0145] The deep learning model performs prediction and outputs three-dimensional distribution of dose.

[0146] A radiotherapy plan generation method of the present embodiment inputs the user's CT data and organ delineation data into the dose prediction model. The dose prediction model is a hybrid UNet model constructed based on the UNet model. The encoding part and the decoding part are respectively introduced into the hybrid module. The hybrid module has multiple branch paths. In this way, when using the dose prediction model to predict the radiotherapy dose distribution, the multiple branch paths of the hybrid module can perform different convolution operations on the input data to obtain different representations, and then add the convolution results of each branch to output, so as to achieve fast and accurate radiotherapy dose distribution prediction and generate a radiotherapy plan.

[0147] Furthermore, the radiotherapy plan generation method of this embodiment guides the optimization based on the predicted three-dimensional dose distribution, optimizes the final plan, and completes the plan generation; specifically includes: calculating the DVH of each organ based on the three-dimensional dose distribution output by the dose prediction model, and setting DVH constraints according to the DVH of each organ; using an optimization algorithm to optimize the three-dimensional dose distribution to obtain a radiotherapy plan.

[0148] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described in the present invention. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the above specific implementation methods. Therefore, any modification or equivalent replacement of the present invention; and all technical solutions and improvements thereof that do not depart from the spirit and scope of the invention are included in the scope of the claims of the present invention.

Claims

1. A radiotherapy intelligent planning method, characterized in that: include: Input the patient's radiotherapy plan input information and predict the patient's three-dimensional dose distribution through the dose prediction model; Generate an initial radiotherapy plan based on the patient's radiotherapy plan input information and three-dimensional dose distribution; The deep Q network model is used to automatically learn and tune the parameters of the initial radiotherapy plan to obtain the final radiotherapy plan that meets clinical requirements.

2. A radiotherapy intelligent planning method according to claim 1, characterized in that: The automatic learning parameter tuning for the initial radiotherapy plan by using the deep Q network model includes: The DVH curve is obtained through three-dimensional dose distribution, and the key ROI and its corresponding dose volume constraint are extracted from the DVH curve; Adjustment actions are set for the dose volume constraints of each key ROI: limit value is increased by a fixed ratio, limit value remains unchanged, limit value is decreased by a fixed ratio, and weight is increased by a fixed ratio, weight remains unchanged, and weight is decreased by a fixed ratio to obtain a discrete action set.

3. A radiotherapy intelligent planning method according to claim 2, characterized in that: The automatic learning parameter tuning is performed for the initial radiotherapy plan by using a deep Q network model, wherein the deep Q network model comprises: Using the embedded dose prediction model to construct the value function q * (s,a) estimation model Q(s,a;W), where s represents the state, a represents the action, and W represents the model parameters; The estimation model Q(s, a; W) includes: a) Input: The input is dose distribution. The initial input is the three-dimensional dose distribution predicted by the dose prediction model, and the subsequent input is the three-dimensional dose distribution of the plan optimization. b) Embedding layer, represented by the dose prediction model; c) Linear layer, stack several layers of linear layers to get the output, the output is: 1×discrete action set size, representing the Q value of each action, and select the action with the maximum Q value as the current action.

4. A radiotherapy intelligent planning method according to claim 3, characterized in that: The automatic learning parameter tuning is performed for the initial radiotherapy plan by using a deep Q network model, wherein the deep Q network model comprises: Define the score function For each key ROI, the score is defined as a continuous function between 0 and 1, and the sum of the scores is calculated to obtain the reward function 5. A radiotherapy intelligent planning method according to claim 4, characterized in that: The automatic learning parameter tuning is performed for the initial radiotherapy plan by using a deep Q network model, wherein the deep Q network model comprises: The loss function is defined by combining experience replay and fixed Q target: Among them, γ is the discount factor. The specific practice of experience replay is to maintain a replay buffer, store the four-element array (state, action, reward, next state) sampled from the environment each time into the replay buffer, and randomly extract several samples from the replay buffer for training when training the deep Q network model. Keep it relatively fixed during the training process and update it to the latest network parameters every certain training cycle:

6. A radiotherapy intelligent planning method according to claim 5, characterized in that: The automatic learning parameter tuning is performed for the initial radiotherapy plan by using the deep Q network model, wherein the training process of the deep Q network model includes: Collect historical patient tumor precision radiotherapy planning data and divide it into training set and validation set. The training set includes CT data and key ROI delineation data, and the validation set includes the final radiotherapy plan. Use the trained dose prediction model to predict the training set data to obtain the three-dimensional dose distribution; The DVH curve is obtained through the three-dimensional dose distribution, and the key ROI and its corresponding dose volume constraint are extracted from the DVH curve. The adjustment action is set for the dose volume constraint of each key ROI to obtain a discrete action set; Using the embedded dose prediction model to construct the value function q * The estimation model Q(s,a;W) of (s,a) is initially inputted as the three-dimensional dose distribution predicted by the MB-UNet model, and then as the three-dimensional dose distribution of the planned optimization. The output is: 1×discrete action set size, which represents the Q value of each action. The action with the maximum Q value is selected as the current action. Define the score function using piecewise function For each key ROI, the score is defined between 0 and 1. The segmentation threshold is set by a senior physicist. The total score is calculated to obtain the reward function. The model is trained by combining experience replay and fixed Q target to define the loss function. Among them, γ is the discount factor. When training the deep Q network model, several samples are randomly extracted from the playback buffer for training. At the same time, the deep Q network model During the training process, the parameters are kept relatively fixed and updated to the latest network parameters every preset number of rounds: When training the deep Q network model, the maximum number of iterations is set for each patient data, and the maximum number of iterations of each training set data is completed in turn: that is, the initialization scene s 0 For the three-dimensional dose distribution predicted by the dose prediction model, the ∈-greedy strategy is used to select action a 0 , using a 0 Update the constraints and then call the radiotherapy planning system optimization interface to obtain the dose distribution, that is, the next scenario s 1 , calculate the reward function r 0 , and finally store the sample (s) in the playback buffer 0 ,a 0 ,r 0 ,s 1 ), and finally the deep Q network model parameters are trained by the experience replay strategy.

7. A radiotherapy intelligent planning method according to claim 6, characterized in that: The automatic learning parameter tuning of the initial radiotherapy plan by the deep Q network model to obtain the final radiotherapy plan that meets clinical requirements includes: When the deep Q network model is trained, a new patient data is input; The three-dimensional dose distribution is predicted by the dose prediction model, and the action a is selected using the trained deep Q network model. 0 , and then obtain the reward function r 0 , continue and get the reward function r 1 ,r 2 …r n , when n reaches the set value, the optimization ends and the final plan is obtained.

8. A radiotherapy intelligent planning method according to any one of claims 1 to 7, characterized in that: The radiotherapy plan input information inputted to the patient includes: patient CT, target area and key ROI delineation, and standardization processing is performed on the radiotherapy plan input information; After the radiotherapy plan input information is standardized, all radiotherapy plan input information, distance features extracted from the radiotherapy plan input information, and beam mask features extracted from the target area are converted into fixed sizes and then combined into inputs of the dose prediction model.

9. A radiotherapy intelligent planning method according to any one of claims 1 to 7, characterized in that: The dose prediction model uses a multi-branch UNet model (MB-UNet), which has a "U"-shaped structure and includes a coding layer, a decoding layer, and a skip connection layer.

10. A radiotherapy planning system for implementing the radiotherapy intelligent planning method according to any one of claims 1 to 9, characterized in that: include: The pre-processing and input module inputs the patient’s radiotherapy plan input information; The dose prediction module predicts the patient's three-dimensional dose distribution through the dose prediction model, and generates an initial radiotherapy plan based on the patient's radiotherapy plan input information and the three-dimensional dose distribution; The automatic parameter optimization module uses a deep Q network model to automatically learn and tune the parameters of the initial radiotherapy plan to obtain the final radiotherapy plan that meets clinical requirements.

Images