Dose prediction method and device based on agent, equipment and storage medium
By dynamically adjusting the input data weight and quantitative uncertainty of the agent, combined with reinforcement learning optimization model hyperparameters, the uncertainty management and personalized processing problems in dose prediction are solved, and efficient and reliable dose prediction is achieved.
Patent Information
- Application Number
- CN202510438900.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-08-01
AI Technical Summary
The existing dose prediction methods lack uncertainty management, insufficient personalized processing capabilities, poor adaptability and difficult to explain the predicted results, resulting in increased treatment risks and poor treatment effects.
The agent-based dose prediction method is adopted to dynamically adjust the input data weights through Foundation Model, and the BNN quantification uncertainty is approximate to MC Dropout, and the model hyperparameters are optimized by reinforcement learning to provide dose distribution and uncertainty information.
It improves the accuracy and interpretability of dose prediction, enhances the model's adaptability to individual differences, reduces the risk of treatment, shortens the time for radiotherapy planning, and improves the reliability and safety of predictions.
Smart Images
Figure CN120393306A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of radiotherapy, and particularly relates to an agent-based dose prediction method, device, equipment and storage medium. Background Art
[0002] In Radiation Therapy (RT), dose prediction is one of the important steps in formulating personalized radiotherapy plans. Its core goal is to minimize the damage to surrounding normal tissues while precisely killing tumor cells. Usually, dose prediction needs to combine multimodal data such as the patient's image data (such as CT, CBCT), organ segmentation results, treatment plan information, etc., to generate the radiation dose distribution in three-dimensional space.
[0003] Traditional dose prediction methods mainly include physical model-based calculation methods (such as Monte Carlo simulation, experience-based dose calculation algorithms) and deep learning-based prediction methods. Although physical model methods have high accuracy, they have large computational amounts and long time consumption, making it difficult to meet the requirements of efficient clinical applications. Deep learning-based methods (such as Convolutional Neural Network CNN, Transformer, etc.) can significantly improve the prediction speed, but usually only provide a single deterministic prediction. When encountering edge cases or out-of-distribution samples, it is easy to generate uncontrollable prediction errors. In addition, such methods usually use models with fixed weights and cannot be dynamically adjusted according to specific patient data, resulting in insufficient personalized processing capabilities.
[0004] In actual clinical applications, the uncertainty management of dose prediction models and the interpretability of prediction results have also received increasing attention. For example, when there is a large uncertainty in the dose prediction of a model in a specific area, doctors can conduct key reviews or adjust treatment plans for that area to reduce treatment risks. However, most current deep learning models lack an effective uncertainty quantification mechanism, which makes it difficult for doctors to accurately evaluate the reliability of model prediction results, may lead to too high or too low treatment doses, and thus affect the treatment effect and patient safety.
[0005] In summary, the existing dose prediction methods have the following problems:
[0006] First, the lack of uncertainty management. Traditional deep learning models usually provide a single deterministic prediction and cannot quantify the uncertainty of prediction results. In radiotherapy, this may cause doctors to be unable to accurately evaluate the reliability of the predicted dose, increasing treatment risks.
[0007] Second, the lack of personalized processing capabilities. Existing dose prediction models mostly use fixed weights to process input data, making it difficult to adapt to the individual differences of different patients, and may have prediction biases in special cases or complex anatomical structures.
[0008] Third, the model has poor adaptability. After most models are trained, their weights remain static and cannot be dynamically adjusted according to new input data, resulting in suboptimal performance in specific patients or special treatment scenarios.
[0009] Fourth, it is difficult to interpret the prediction results. Traditional methods only provide predicted values of dose distribution, lacking uncertainty information and risk warnings, making it difficult for doctors to judge the credibility of the prediction results and reducing the safety of clinical applications. Summary of the Invention
[0010] To solve the above technical problems, the present invention proposes an agent-based dose prediction method, device, equipment, and storage medium.
[0011] To achieve the above object, the technical solution of the present invention is as follows:
[0012] In the first aspect, the present invention discloses an agent-based dose prediction method, including:
[0013] Step S1: Collect multi-modal data and preprocess it;
[0014] The multi-modal data includes: medical image data and clinical data;
[0015] Step S2: Input the collected multi-modal data into the dose prediction agent and perform the following steps:
[0016] Step S2.1: The agent uses the Foundation Model to learn the importance distribution of the input data, dynamically adjusts the weights of the input data through the attention mechanism, and obtains new input data;
[0017] Step S2.2: The agent uses MC Dropout to approximate BNN, performs inference using the new input data, introduces randomness during the inference process, and obtains the predicted dose distribution through multiple forward propagations to quantify the prediction uncertainty;
[0018] Step S3: The dose prediction agent outputs the predicted dose distribution and uncertainty information;
[0019] Step S4: Evaluate the output result of the dose prediction agent and feedback the evaluation result to the agent. The agent adaptively adjusts the hyperparameters or inference strategies of the dose prediction model based on reinforcement learning to optimize the prediction performance of the model.
[0020] Based on the above technical solution, the following improvements can also be made:
[0021] As a preferred solution, step S2.1 includes:
[0022] Step S2.1.1: Use Foundation Model to calculate the importance weight of input data X;
[0023] w i =softmax(F(X i ));
[0024] in:
[0025] X is the input data, X={X1,X2,…,X n};
[0026] F(·) is the feature extraction function of Foundation Model;
[0027] w i is the dynamic weight of the input data;
[0028] Step S2.1.2: Based on the dynamic weight w i , get new input data X';
[0029]
[0030] As a preferred solution, step S2.2 includes:
[0031] Step S2.2.1: Apply Dropout to the key layers of the agent’s neural network, i.e., each neuron is randomly set to zero with probability p:
[0032]
[0033] in:
[0034] W is the weight of the network;
[0035] M~Bernoulli(p) is the random mask generated by Dropout;
[0036] f(X′,W⊙M) is the dose prediction result obtained by forward propagation;
[0037] Step S2.2.2: Perform T forward propagations to obtain multiple dose prediction values:
[0038] Step S2.2.3: Calculate the mean dose prediction value based on multiple dose prediction values as follows:
[0039] Step S2.2.4: Calculate the uncertainty based on the mean dose prediction as follows:
[0040] Step S2.2.5: Compare σ p with a preset uncertainty threshold τ. If σ p > τ, mark this area.
[0041] As an optimal solution, the agent controls the entire process of dose prediction, with the goal of maximizing the accuracy of dose prediction, minimizing uncertainty, and optimizing individual adaptability:
[0042]
[0043] where: λ is the weight controlling the uncertainty penalty.
[0044] As an optimal solution, in the reinforcement learning method, the state, action, and reward function are as follows:
[0045] State: The anatomical information S of the current patient t ={X′, OAR, Dose {t-1}};
[0046] Action: Adjust the hyperparameters A of the model t ={θ loss , θ lr , θ arch};
[0047] where:
[0048] θ loss is the loss function parameter;
[0049] θ lr is the learning rate parameter;
[0050] θ arch is the model architecture parameter;
[0051] Reward function: Calculate the reward based on the dose prediction error:
[0052]
[0053] where:
[0054] ROI is the target area, and are the predicted dose and the true dose respectively;
[0055] Use the Deep Deterministic Policy Gradient or PPO algorithm to optimize the decision of the agent:
[0056]
[0057] where α is the learning rate, is the policy gradient.
[0058] In a second aspect, the present invention discloses an agent-based dose prediction device, comprising:
[0059] An acquisition module, configured to acquire multimodal data and preprocess it;
[0060] The multimodal data includes: medical image data and clinical data;
[0061] An input module, configured to input the acquired multimodal data into the dose prediction agent, including:
[0062] An input adjustment unit, configured to enable the agent to learn the importance distribution of the input data using the Foundation Model, and dynamically adjust the weights of the input data through the attention mechanism to obtain new input data;
[0063] An uncertainty management unit, configured to enable the agent to approximate the BNN using MC Dropout, perform inference using the new input data, introduce randomness during the inference process, and obtain the predicted dose distribution through multiple forward propagations to quantify the prediction uncertainty;
[0064] An output module, configured to output the predicted dose distribution and uncertainty information by the dose prediction agent;
[0065] A feedback optimization module, configured to evaluate the output result of the dose prediction agent and feedback the evaluation result to the agent, and the agent adaptively adjusts the hyperparameters or inference strategy of the dose prediction model based on reinforcement learning to optimize the prediction performance of the model.
[0066] As a preferred solution, the input adjustment unit includes:
[0067] A weight calculation unit, configured to calculate the importance weights of the input data X using the Foundation Model;
[0068] w i = softmax(F(X i ));
[0069] Where:
[0070] X is the input data, X = {X1, X2, …, X n};
[0071] F(·) is the feature extraction function of the Foundation Model;
[0072] w i is the dynamic weight of the input data;
[0073] The input adjustment unit, configured to be based on the dynamic weight w i, obtain new input data X';
[0074]
[0075] As a preferred solution, the uncertainty management unit includes:
[0076] A mechanism application unit, which is used to apply Dropout at the key layers of the agent neural network, that is, each neuron is randomly set to zero with probability p:
[0077]
[0078] Where:
[0079] W is the weight of the network;
[0080] M ~ Bernoulli(p) is the random mask generated by Dropout;
[0081] f(X′, W⊙M) is the dose prediction result obtained by forward propagation;
[0082] A forward propagation unit, which is used to perform forward propagation T times to obtain multiple dose prediction values:
[0083] A mean calculation unit, which is used to calculate the dose prediction mean based on multiple dose prediction values As follows:
[0084]
[0085] An uncertainty calculation unit, which is used to calculate the uncertainty based on the dose prediction mean, specifically as follows:
[0086]
[0087] A marking unit, which is used to compare σ p with a preset uncertainty threshold τ. If σ p > τ, then mark this area.
[0088] As a preferred solution, the agent controls the whole process of dose prediction, with the goal of maximizing the accuracy of dose prediction, minimizing uncertainty, and optimizing individual adaptability:
[0089]
[0090] Where: λ is the weight for controlling the uncertainty penalty.
[0091] As a preferred solution, in the reinforcement learning method, the state, action, and reward function are as follows:
[0092] Status: The anatomical information S of the current patient t ={X′, OAR, Dose {t-1}};
[0093] Action: Adjust the hyperparameters A of the model t ={θ loss , θ lr , θ arch};
[0094] Wherein:
[0095] θ loss is the loss function parameter;
[0096] θ lr is the learning rate parameter;
[0097] θ arch is the model architecture parameter;
[0098] Reward function: Calculate the reward based on the dose prediction error:
[0099]
[0100] Wherein:
[0101] ROI is the target area, and are the predicted dose and the true dose respectively;
[0102] Use the deep deterministic policy gradient or PPO algorithm to optimize the decision of the agent:
[0103]
[0104] Where α is the learning rate, is the policy gradient.
[0105] Thirdly, the present invention discloses a computing device, comprising:
[0106] One or more processors;
[0107] A memory;
[0108] And one or more programs, wherein one or more programs are stored in the memory and are configured to be executed by one or more processors, and one or more programs include instructions for any of the above agent-based dose prediction methods.
[0109] Fourthly, the present invention discloses a storage medium, and the storage medium stores one or more computer-readable programs, and one or more programs include instructions, and the instructions are adapted to be loaded and executed by the memory for any of the above agent-based dose prediction methods.
[0110] The present invention discloses an agent-based dose prediction method, device, equipment and storage medium, which can effectively solve the problems of insufficient uncertainty management, personalized processing, self-adaptability and interpretability of prediction results in the prior art in dose prediction, and has the following beneficial effects:
[0111] First, the present invention uses MC Dropout to approximate Bayesian neural network (BNN), providing uncertainty assessment for prediction results while maintaining efficient computation, helping doctors identify high-risk areas and improving treatment safety.
[0112] Second, the present invention introduces the Foundation Model to achieve dynamic weighted processing of different modality input data, enhancing the model's adaptability to individual differences of different patients.
[0113] Third, the present invention uses reinforcement learning method to optimize the dose prediction model, dynamically adjusting the model hyperparameters according to the actual input data to ensure the best prediction effect of the model under specific patients or special treatment conditions.
[0114] Fourth, the present invention combines uncertainty information, providing the mean and variance of the prediction distribution to provide more references for doctors' decision-making and reducing potential risks during the treatment process.
[0115] Fifth, through an automated and intelligent dose prediction method, the workload of manual tuning and review is reduced, the radiotherapy plan formulation time is shortened, and the reliability and accuracy of prediction are improved at the same time. Description of the Drawings
[0116] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.
[0117] Figure 1 It is a flowchart of the dose prediction method provided by the embodiment of the present invention. Detailed Embodiments
[0118] The preferred embodiments of the present invention will be described in detail below with reference to the drawings.
[0119] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0120] The expression "comprising" an element is an "open-ended" expression, which merely means that the corresponding components or steps exist and should not be construed as excluding additional components or steps.
[0121] To achieve the objectives of the present invention, in some embodiments of the agent-based dose prediction method, as Figure 1 shown, the method comprises:
[0122] Step S101: Collect multimodal data and preprocess it;
[0123] The multimodal data includes: medical image data and clinical data;
[0124] Step S102: Input the collected multimodal data into the dose prediction agent and perform the following steps:
[0125] Step S102.1: The agent uses the Foundation Model to learn the importance distribution of the input data, dynamically adjusts the weights of the input data through the attention mechanism, and obtains new input data;
[0126] Step S102.2: The agent uses MC Dropout to approximate BNN, performs inference using the new input data, introduces randomness during the inference process, and obtains the predicted dose distribution through multiple forward propagations to quantify the prediction uncertainty;
[0127] Step S103: The dose prediction agent outputs the predicted dose distribution and uncertainty information;
[0128] Step S104: Evaluate the output result of the dose prediction agent, feedback the evaluation result to the agent, and the agent adaptively adjusts the hyperparameters or inference strategy of the dose prediction model based on reinforcement learning to optimize the prediction performance of the model.
[0129] The above steps will be elaborated in detail below.
[0130] Step S101 is the step of collecting and preprocessing multimodal data.
[0131] The multimodal data includes: medical image data (such as: CT images, CBCT images, plain film X-rays, etc.) and clinical data (such as: segmentation results, clinical information and other auxiliary data). The preprocessing operations include but are not limited to: data standardization, spatial registration, and data augmentation processing, etc.
[0132] Step S102: Input the collected multimodal data into the dose prediction agent.
[0133] Traditional dose prediction usually fixes the weights of input data such as CT, OAR segmentation, tumor region, etc., without considering the individual differences of different patients. The present invention uses a Foundation Model to learn the importance distribution of input data, and dynamically adjusts the weights of input data through an attention mechanism to improve the personalized prediction ability.
[0134] Specifically, step S102.1 includes:
[0135] Step S102.1.1: Calculate the importance weights of input data X using a Foundation Model;
[0136] w i = softmax(F(X i ));
[0137] Where:
[0138] X is the input data, X = {X1, X2, …, X n}, such as: CT, CBCT, OAR segmentation, etc.;
[0139] F(·) is the feature extraction function of the Foundation Model;
[0140] w i is the dynamic weight of the input data;
[0141] Step S2.1.2: Based on the dynamic weight w i , obtain the new input data X';
[0142]
[0143] The Foundation Model extracts multi-modal data features, dynamically assigns weights to input data according to patient characteristics, realizes personalized feature extraction, and improves the personalized ability of dose prediction. Traditional methods fix input data, and this method enables the agent to dynamically adjust the weights of input data to improve individual adaptability.
[0144] Traditional dose prediction models output a single dose value, unable to quantify the uncertainty of the prediction, which may lead to underdose or overdose.
[0145] The present invention uses Monte Carlo Dropout (MC Dropout) to approximate Bayesian neural network (BNN), calculates the confidence interval through multiple random samplings, enabling doctors to intuitively evaluate the reliability of the prediction.
[0146] Specifically, step S102.2 includes:
[0147] Step S102.2.1: Apply Dropout to the key layers (such as fully connected layers and convolutional layers) of the agent neural network, that is, each neuron is randomly set to zero with a certain probability p:
[0148]
[0149] Where:
[0150] W is the weight of the network;
[0151] M ~ Bernoulli(p) is the random mask generated by Dropout;
[0152] f(X′, W⊙M) is the dose prediction result obtained by forward propagation;
[0153] Step S102.2.2: Perform T times of forward propagation to obtain multiple dose prediction values:
[0154] Step S102.2.3: Calculate the dose prediction mean based on multiple dose prediction values As follows:
[0155]
[0156] Step S102.2.4: Calculate the uncertainty based on the dose prediction mean, specifically as follows:
[0157]
[0158] Step S102.2.5: Compare σ p with the preset uncertainty threshold τ. If σ p > τ, then mark this area.
[0159] Traditional methods cannot quantify uncertainty. The present invention uses MC Dropout to approximate BNN, performs multiple forward propagations during the inference process, calculates the mean and variance of the prediction distribution, and quantifies the prediction uncertainty. The present invention enables the agent to provide a confidence interval, enhancing the interpretability and safety of dose prediction.
[0160] The agent controls the whole process of dose prediction. The goal is to maximize the accuracy of dose prediction, minimize uncertainty, and optimize individual adaptability:
[0161]
[0162] Where: λ is the weight controlling the uncertainty penalty, ensuring high precision and stability.
[0163] Step S103 outputs the predicted dose distribution and uncertainty information.
[0164] Output the predicted dose distribution (such as DVH curve, spatial dose distribution map) and uncertainty information (such as confidence interval).
[0165] Generate a visual prediction report to provide auxiliary decision-making support for doctors.
[0166] Interact with other radiotherapy systems (such as TPS) through a data interface.
[0167] Step S104 is a feedback and adaptive optimization step.
[0168] Obtain feedback information through doctor evaluation and actual treatment effects (such as subsequent imaging data).
[0169] The reinforcement learning agent adjusts the behavior of the model through a reward mechanism (Reward) to achieve adaptive optimization.
[0170] Traditional dose prediction uses fixed model parameters and cannot be adaptively adjusted according to patient characteristics. The present invention adopts the reinforcement learning method to enable the agent to automatically optimize the hyperparameters of the dose prediction model, such as the weights of the loss function, the learning rate, the model structure, etc.
[0171] In step S104, in the reinforcement learning method, the state, action, and reward function are as follows:
[0172] State: The anatomical information S of the current patient t ={X′, OAR, Dose {t-1}};
[0173] Action: Adjust the hyperparameters A of the model t ={θ loss , θ lr , θ arch};
[0174] Where:
[0175] θ loss Is the loss function parameter, used to control the weights of different loss terms in model training;
[0176] θ lr Is the learning rate parameter, used to control the step size of model parameter update;
[0177] θ arch Is the model architecture parameter, including parameters related to the network structure, such as the number of layers, the number of neurons, the selection of activation functions, etc.;
[0178] Reward function (Reward): Calculate the reward based on the dose prediction error:
[0179]
[0180] Wherein:
[0181] ROI is the target region, and are the predicted dose and the true dose respectively;
[0182] To guide the agent to learn the optimal policy to maximize the long-term cumulative reward, reinforcement learning algorithms usually introduce a value function (Value Function) to estimate future returns. Specifically:
[0183] The state value function V(S_t) is used to evaluate the expected total reward that the agent can obtain by following a specific policy in the current state S_t.
[0184] The action value function Q(S_t, A_t) is used to evaluate the expected total reward that the agent can obtain by following a specific policy after executing the action A_t in the state S_t.
[0185] The agent interacts with the environment (i.e., makes dose predictions and obtains the reward R t ), and continuously learns and updates the estimation of the value function.
[0186] Subsequently, based on the learned value function, advanced reinforcement learning algorithms such as Deep Deterministic Policy Gradient (DDPG) or Proximal Policy Optimization (PPO) are used to optimize the decision-making policy θ of the agent (the policy network for adjusting the model hyperparameters), with the goal of maximizing the expected return E[R t . Policy updates usually rely on the policy gradient whose calculation often depends on the estimation of the value function, and the formula is as follows:
[0187]
[0188] Wherein: α is the learning rate, J(θ) is the policy performance objective function, and its gradient whose calculation depends on the estimation of the value function.
[0189] The present invention dynamically adjusts the hyperparameters or inference strategies of the model according to environmental feedback (such as doctor evaluation, actual treatment effect) through the reinforcement learning (RL) method, and continuously optimizes the prediction performance.
[0190] In summary, the present invention has the following advantages:
[0191] First, the present invention approximates Bayesian neural networks (BNNs) through MC Dropout, introduces randomness during the inference phase, and obtains the predictive distribution through multiple forward propagations, thereby quantifying the uncertainty of predictions. Doctors can evaluate the safety of treatment plans based on the mean and variance of the prediction results, especially providing more decision-making references in high-risk areas.
[0192] Second, the present invention introduces a Foundation Model, which dynamically adjusts the weights of different input features through an agent, enabling the model to adaptively select key features among different patients. This not only improves the accuracy of predictions but also provides more targeted auxiliary decision-making support for doctors, achieving personalized radiotherapy.
[0193] Third, through reinforcement learning (RL) technology, the present invention designs the dose prediction model as an agent, and dynamically adjusts the model hyperparameters through a feedback mechanism. When the prediction results are not satisfactory, the agent can continuously self-learn and optimize based on environmental feedback (such as doctors' evaluations and actual treatment effects), ensuring that the model always maintains the best prediction state.
[0194] Fourth, by combining uncertainty information with the reinforcement learning mechanism, the present invention improves the interpretability of prediction results and decision-making support capabilities. Through the predictive distribution generated by MC Dropout, the agent not only provides the prediction mean but also shows the confidence interval of the predictive distribution, enabling doctors to intuitively evaluate the prediction confidence of the model in different regions. At the same time, by analyzing the value functions learned during the reinforcement learning process (for example, the state value function V(S t ) or the action value function Q(S t , A t ), it is possible to identify which states (i.e., specific combinations of patient anatomical information and historical doses) are evaluated by the agent as being more likely to lead to high rewards (i.e., low prediction errors), or which hyperparameter adjustment actions A t are considered optimal in a specific state S t . This helps to reveal part of the decision-making logic of the model, identify the factors that have the greatest impact on the final prediction performance, and thus improve the transparency of the model and the credibility of clinical applications.
[0195] Fifth, through the automated and intelligent decision-making capabilities of the agent, the present invention can provide accurate and reliable dose prediction results in a short time. In areas with high uncertainty, through the confidence interval provided by the model and the adaptive tuning of reinforcement learning, doctors can more efficiently locate potential risk areas, achieve rapid verification and correction, and significantly improve clinical efficiency.
[0196] In some other embodiments, the present invention discloses an agent-based dose prediction device, including:
[0197] A data acquisition module for acquiring multi-modal data and preprocessing it;
[0198] The multi-modal data includes: medical image data and clinical data;
[0199] An input module for inputting the acquired multi-modal data into the dose prediction agent, including:
[0200] An input adjustment unit for the agent to learn the importance distribution of the input data using the Foundation Model, dynamically adjust the weights of the input data through the attention mechanism, and obtain new input data;
[0201] An uncertainty management unit for the agent to approximate the BNN using MC Dropout, perform inference using the new input data, introduce randomness during the inference process, and obtain the predicted dose distribution through multiple forward propagations to quantify the prediction uncertainty;
[0202] An output module for the dose prediction agent to output the predicted dose distribution and uncertainty information;
[0203] A feedback optimization module for evaluating the output results of the dose prediction agent and feeding back the evaluation results to the agent. The agent adaptively adjusts the hyperparameters or inference strategies of the dose prediction model based on reinforcement learning to optimize the prediction performance of the model.
[0204] As a preferred solution, the input adjustment unit includes:
[0205] A weight calculation unit for calculating the importance weights of the input data X using the Foundation Model;
[0206] w i = softmax(F(X i ));
[0207] Where:
[0208] X is the input data, X = {X1, X2, …, X n};
[0209] F(·) is the feature extraction function of the Foundation Model;
[0210] w i is the dynamic weight of the input data;
[0211] The input adjustment unit for obtaining new input data X' based on the dynamic weight w i ;
[0212]
[0213] As a preferred solution, the uncertainty management unit includes:
[0214] A mechanism application unit, which is used to apply Dropout at the key layers of the agent neural network, that is, each neuron is randomly set to zero with probability p:
[0215]
[0216] Where:
[0217] W is the weight of the network;
[0218] M ∼ Bernoulli(p) is the random mask generated by Dropout;
[0219] f(X′, W ⊙ M) is the dose prediction result obtained by forward propagation;
[0220] A forward propagation unit, which is used to perform T times of forward propagation to obtain multiple dose prediction values:
[0221] A mean calculation unit, which is used to calculate the dose prediction mean based on multiple dose prediction values As follows:
[0222]
[0223] An uncertainty calculation unit, which is used to calculate the uncertainty based on the dose prediction mean, specifically as follows:
[0224]
[0225] A marking unit, which is used to compare σ p with a preset uncertainty threshold τ. If σ p > τ, then mark this area.
[0226] Furthermore, the agent controls the whole process of dose prediction, with the goal of maximizing the accuracy of dose prediction, minimizing uncertainty, and optimizing individual adaptability:
[0227]
[0228] Where: λ is the weight for controlling the uncertainty penalty.
[0229] Furthermore, in the reinforcement learning method, the state, action, and reward function are as follows:
[0230] State: The anatomical information S of the current patient t ={X′, OAR, Dose {t-1}};
[0231] Action: Adjust the hyperparameter A of the model t ={θ loss ,θ lr ,θ arch};
[0232] Wherein:
[0233] θ loss is the loss function parameter;
[0234] θ lr is the learning rate parameter;
[0235] θ arch is the model architecture parameter;
[0236] Reward function: Calculate the reward based on the dose prediction error:
[0237]
[0238] Wherein:
[0239] ROI is the target area, and are the predicted dose and the true dose respectively;
[0240] Use the deep deterministic policy gradient or PPO algorithm to optimize the agent's decision:
[0241]
[0242] Where α is the learning rate, is the policy gradient.
[0243] Furthermore, it should be noted that: When the agent-based dose prediction device provided in the above embodiments performs dose prediction, only the above-mentioned division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the agent-based dose prediction device is divided into different functional modules to complete all or part of the functions described above.
[0244] In addition, the agent-based dose prediction device provided in the above embodiments and the embodiments of the agent-based dose prediction method belong to the same concept. For the specific implementation process, please refer to the method embodiments and will not be elaborated here.
[0245] In some other embodiments, the present invention discloses a computing device, including:
[0246] One or more processors;
[0247] Memory;
[0248] and one or more programs, wherein the one or more programs are stored in a memory and configured to be executed by one or more processors, and the one or more programs include instructions for any of the above agent-based dose prediction methods.
[0249] In some other embodiments, the present invention discloses a storage medium storing one or more computer-readable programs, the one or more programs including instructions adapted to be loaded and executed by a memory for any of the above agent-based dose prediction methods.
[0250] The present invention discloses an agent-based dose prediction method, device, equipment and storage medium, which can effectively solve the problems of uncertainty management, personalized processing, self-adaptability and insufficient interpretability of prediction results in the prior art in dose prediction, and has the following beneficial effects:
[0251] First, the present invention uses MC Dropout to approximate Bayesian neural network (BNN), providing uncertainty assessment for prediction results while maintaining efficient computation, helping doctors identify high-risk areas and improving treatment safety.
[0252] Second, the present invention introduces a Foundation Model to achieve dynamic weighted processing of different modal input data, enhancing the model's adaptability to individual differences among different patients.
[0253] Third, the present invention uses a reinforcement learning method to optimize the dose prediction model, dynamically adjusting the model hyperparameters according to the actual input data to ensure the best prediction effect of the model under specific patients or special treatment conditions.
[0254] Fourth, the present invention combines uncertainty information, providing the mean and variance of the prediction distribution to provide more references for doctors' decisions and reducing potential risks during the treatment process.
[0255] Fifth, through an automated and intelligent dose prediction method, the workload of manual parameter adjustment and review is reduced, the radiotherapy plan formulation time is shortened, and the reliability and accuracy of the prediction are improved at the same time.
[0256] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification only illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of the present invention claimed is defined by the appended claims and their equivalents.
Claims
1. Agent-based dose prediction method, characterized in that Including: Step S1: Collect multimodal data and preprocess it; The multimodal data includes: medical image data and clinical data; Step S2: Input the collected multimodal data into the dose prediction agent and perform the following steps: Step S2.1: The agent uses the Foundation Model to learn the importance distribution of the input data, dynamically adjusts the weights of the input data through the attention mechanism, and obtains new input data; Step S2.2: The agent uses MC Dropout to approximate BNN, performs inference using the new input data, introduces randomness during the inference process, and obtains the predicted dose distribution through multiple forward propagations to quantify the prediction uncertainty; Step S3: The dose prediction agent outputs the predicted dose distribution and uncertainty information; Step S4: Evaluate the output result of the dose prediction agent, and feedback the evaluation result to the agent. The agent adaptively adjusts the hyperparameters or inference strategy of the dose prediction model based on reinforcement learning to optimize the prediction performance of the model.
2. The dosage prediction method according to claim 1, wherein The said Step S2.1 includes: Step S2.1.1: Use the Foundation Model to calculate the importance weights of the input data X; w i = softmax(F(X i )); Where: X is the input data, X = {X1, X2, …, X n}; F(·) is the feature extraction function of the Foundation Model; w i is the dynamic weight of the input data; Step S2.1.2: Based on the dynamic weight w i , obtain the new input data X'; 3. The dose prediction method according to claim 2, wherein The said Step S2.2 includes: Step S2.2.1: Apply Dropout at the key layers of the agent neural network, that is, each neuron is randomly set to zero with probability p: Where: W is the weight of the network; M~Bernoulli(p) is the random mask generated by Dropout; f(X',W⊙M) is the dose prediction result obtained by forward propagation; Step S2.2.2: Perform T times of forward propagation to obtain multiple dose prediction values; Step S2.2.3: Calculate the mean dose prediction based on multiple dose prediction values As follows: Step S2.2.4: Calculate the uncertainty based on the dose prediction mean, specifically as follows: Step S2.2.5: Compare σ p with a preset uncertainty threshold τ. If σ p > τ, mark this area.
4. The dose prediction method according to claim 3, wherein The agent controls the whole process of dose prediction, and the goal is to maximize the accuracy of dose prediction, minimize the uncertainty, and optimize the individual adaptability: Where: λ is the weight for controlling the uncertainty penalty.
5. The dosage prediction method according to claim 3, characterized in that In the reinforcement learning method, the state, action, and reward function are as follows: Status: The anatomical information S of the current patient t ={X', OAR, Dose {t-1}}; Action: Adjust the hyperparameter A of the model t ={θ loss , θ lr , θ arch}; Where: θ loss is the loss function parameter; θ lr is the learning rate parameter; θ arch is a model architecture parameter; Reward function: Calculate the reward based on the dose prediction error: Where: The ROI is the target region, and are the predicted dose and the true dose, respectively; Use the deep deterministic policy gradient or PPO algorithm to optimize the decision of the agent: where α is the learning rate, is the policy gradient.
6. Agent-based dose prediction device, characterized in that Including: A collection module for collecting multimodal data and preprocessing it; The multimodal data includes: medical image data and clinical data; An input module for inputting the collected multimodal data into the dose prediction agent, including: An input adjustment unit for the agent to use the Foundation Model to learn the importance distribution of the input data, dynamically adjust the weights of the input data through the attention mechanism, and obtain new input data; An uncertainty management unit for the agent to use MC Dropout to approximate BNN, perform inference using the new input data, introduce randomness during the inference process, and obtain the predicted dose distribution through multiple forward propagations to quantify the prediction uncertainty; An output module for the dose prediction agent to output the predicted dose distribution and uncertainty information; A feedback optimization module for evaluating the output results of the dose prediction agent and feeding back the evaluation results to the agent. The agent adaptively adjusts the hyperparameters or inference strategies of the dose prediction model based on reinforcement learning to optimize the prediction performance of the model.
7. The dose prediction device according to claim 6, characterized in that, The input adjustment unit includes: A weight calculation unit for calculating the importance weights of the input data X using the Foundation Model; w i = softmax(F(X i )); Where: X is the input data, X = {X1, X2, …, X n}; F(·) is the feature extraction function of the Foundation Model; w i is the dynamic weight of the input data; An input adjustment unit for obtaining new input data X' based on the dynamic weight w i ,; 8. The dose prediction device according to claim 7, characterized in that The uncertainty management unit includes: A mechanism application unit for applying Dropout at the key layers of the agent neural network, i.e., each neuron is randomly set to zero with probability p: Where: W is the weight of the network; M ∼ Bernoulli(p) is the random mask generated by Dropout; f(X', W⊙M) is the dose prediction result obtained by forward propagation; A forward propagation unit for performing T times of forward propagation to obtain multiple dose prediction values; A mean calculation unit for calculating a mean dose prediction based on a plurality of dose prediction values as follows: An uncertainty calculation unit for calculating the uncertainty based on the dose prediction mean, specifically as follows: A marking unit for marking σ p and comparing it with a preset uncertainty threshold τ. If σ p > τ, then mark this area.
9. A computing device, characterized in that, Including: One or more processors; A memory; And one or more programs, where the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for the agent-based dose prediction method according to any one of claims 1-4 above.
10. A storage medium, characterized in that, The storage medium stores one or more computer-readable programs. The one or more programs include instructions that are adapted to be loaded and executed by the memory for the agent-based dose prediction method according to any one of claims 1-4 above.