Automatic prediction method for dose distribution in radiotherapy plan based on improved FiT

Abstract

The invention provides an automatic prediction method and system for dose distribution in a radiotherapy plan based on improved FiT. Comprising the following steps: acquiring a noise image and a to-be-predicted structure image, and respectively preprocessing the noise image and the to-be-predicted structure image; and inputting a preprocessing result into a pre-trained improved FiT model, carrying out reverse denoising on the model under the guidance of a structural image, and carrying out reconstruction decoding on a denoised feature map to obtain a predicted dose distribution map. The improved FiT model comprises a base model and a condition model mounted on the base model, and the base model is a pre-trained FiT model. According to the method, the diffusion model of the Flexible Vision Transform architecture is used, the prediction precision is maintained, the consumption of computing resources is reduced, the efficiency of the radiotherapy plan is improved, and the controllability of model generation is further improved by combining the semantic features of the structural image.

Description

Technical Field

[0001] The present invention relates to the field of radiotherapy preoperative planning, and in particular to an automatic prediction method for dose distribution in radiotherapy planning based on improved FiT. Background Art

[0002] Radiotherapy is one of the mainstream treatments for cancer patients and has made remarkable progress in the past few decades. In order to achieve good treatment results, high-quality radiotherapy plans are required to distribute sufficient radiation dose to the planning target volume (PTV) while minimizing radiation hazards to organs at risk (OARs). To achieve this, radiotherapy plans need to be manually adjusted by dosimetrists in a trial-and-error manner, which is very labor-intensive and time-consuming. In addition, the quality of treatment plans may vary due to different expertise and experience of radiologists.

[0003] Diffusion models have demonstrated their significant potential in modeling complex image distributions in certain vision tasks. Unlike other Deep Learning (DL) models, diffusion models do not make any additional assumptions about the target data distribution during training, thus avoiding the averaging effect and alleviating the over-smoothing problem. The diffusion-based model predicts dose maps with the shape of the ray penetration area and clearer boundaries. Therefore, introducing diffusion models into dose prediction tasks is a worthy endeavor.

[0004] The advantages of Transformer-based networks over convolutional networks have been widely established in various studies, demonstrating properties such as robustness, effective modal fusion and scalability. Similarly, the results of multimodal fusion compared to the CNN-based generator U-Net are consistent with our observations in this study. This advantage is not only due to the high-quality alignment achieved in the second training stage, but also due to the multi-head attention-based fusion mechanism, which is good at modeling long dependencies. This mechanism effectively integrates the compositional semantic information, more effectively guides the generation of visual latent vectors, and produces images that are closely aligned with the input image control. These findings highlight the unique advantages of the Diffusion Transformer architecture in effectively fusing multimodal information.

[0005] Current image generation models have difficulty generalizing across arbitrary resolutions. The Diffusion Transformer (DiT) family, while performing well in a certain range of resolutions, performs poorly when dealing with images of varying resolutions. This limitation stems from DiT's inability to exploit dynamic-resolution images during its training, hampering its ability to effectively adapt to varying token lengths or resolutions. The Flexible Vision Transformer (FiT) conceptualizes images as sequences of dynamically sized tokens. This perspective enables a flexible training strategy that can easily adapt to different aspect ratios during both training and inference phases, thereby promoting resolution generalization and eliminating bias caused by image cropping. Through a carefully tuned network structure and the integration of training-free extrapolation techniques, FiT demonstrates remarkable flexibility in resolution extrapolation generation. Comprehensive experiments demonstrate the superior performance of FiT across a wide range of resolutions.

[0006] In this paper, we explored the feasibility of applying diffusion models to dose prediction tasks and proposed a diffusion-based model to automatically predict clinically acceptable dose distributions for lung cancer patients. Specifically, it consists of a forward process and a reverse process. In the forward process, the model uses a Markov chain to gradually transform the dose distribution, mapping the complex distribution to a Gaussian distribution by gradually adding predefined noise. Then, in the reverse process, given a pure Gaussian noise, the model gradually removes the noise in multiple steps and finally outputs a predicted dose map. In this process, the noise predictor is trained to predict the noise added in the corresponding step of the forward process. To further ensure the accuracy of the predicted dose distribution of PTV and OAR, we designed a DL-based structural encoder to extract anatomical information from CT images and segmentation masks of PTV and OAR. Such anatomical information can indicate the structure and relative position of organs. By incorporating anatomical information, the noise predictor can know the dose constraints between PTV and OAR, thereby allocating more appropriate doses to them and generating more accurate dose distribution maps.

[0007] Therefore, the present invention provides an automatic prediction method for dose distribution in radiotherapy planning based on improved FiT to solve the above problems. Summary of the invention

[0008] In view of the above, the present invention proposes an automatic prediction method for dose distribution in radiotherapy planning based on improved FiT. On the basis of using diffusion model for dose prediction in radiotherapy planning, transformer architecture is introduced to ensure the accuracy of dose prediction while reducing the consumption of computing resources and enhancing the generation capability. Conditions are added to low-dimensional feature information to improve the controllability of model generation.

[0009] To achieve the above-mentioned object of the invention, the present invention proposes an automatic prediction method for dose distribution in radiotherapy planning based on improved FiT, comprising:

[0010] Acquire a noise image and a structural image of the patient's lesion site information to be predicted, and preprocess the noise image and the structural image to be predicted to obtain feature maps of the noise image and the structural image to be predicted;

[0011] Inputting the feature maps of the noise image and the structure image to be predicted into the pre-trained improved FiT model, the improved FiT model performs reverse denoising on the feature map of the noise image under the guidance of the structure image to be predicted, and obtains the feature map of the predicted dose distribution map; the improved FiT model includes a base model and a conditional model mounted on the base model, the base model is a pre-trained FiT model, receives the input noise image, and the conditional model receives the structure image to be predicted;

[0012] The feature map of the predicted dose distribution map is reconstructed and decoded to obtain the predicted dose distribution map.

[0013] Preferably, the pre-training process of the improved FiT model is specifically as follows:

[0014] 1) obtaining a structural image containing information about the patient's lesion location and a real dose distribution map corresponding to the structural image, wherein the structural image containing information about the patient's lesion location includes a CT image of the patient, a planned target volume, and a segmentation mask of an organ at risk;

[0015] 2) Constructing a conditional model, using the pre-trained FiT model as a base model, and mounting the conditional model on the base model to form an improved FiT model;

[0016] 3) Preprocessing the structural image and the true dose distribution map to obtain characteristic maps of the structural image and the dose distribution map respectively;

[0017] 4) Freeze the base model, input the feature map of the structural image into the conditional model, input the feature map of the real dose distribution map into the base model, and train the improved FiT model. During the training process, the improved FiT model forward-noises the feature map of the real dose distribution map under the guidance of the structural image, and uses a noise predictor to predict the noise added in the forward noisy process; the training goal is to minimize the difference between the noise predicted by the noise predictor and the noise actually added by the improved FiT model.

[0018] Preferably, the conditional model comprises N FiT blocks and N+1 zero linear layers, wherein the first zero linear layer is used as the first layer of the conditional model and is serially connected to the N FiT blocks, and the output of each FiT block is processed by a zero linear layer and then output to the base model;

[0019] Preferably, the mounting relationship between the base model and the conditional model is specifically as follows:

[0020] The output of the first zero linear layer in the conditional model is added and combined with the input of the base model as the input of the first FiT block in the conditional model. The output of the i-th FiT block in the conditional model is processed by the zero linear layer and added and combined with the output of the i-th FiT block in the base model as the input of the i+1-th FiT block in the base model. The number of FiT blocks in the conditional model is less than the number of FiT blocks in the base model.

[0021] Preferably, the preprocessing of the noise image and the structure image to be predicted is consistent with the preprocessing method of the structure image and the true dose distribution map in the pre-training process of the improved FiT model. The preprocessing method is specifically:

[0022] For the image to be preprocessed, the encoder part of the variational autoencoder is compressed into a low-dimensional latent space representation. The low-dimensional latent space representation is first patched into multiple tokens of the same size and non-overlapping. Multiple tokens are rearranged into a token sequence, and then the token sequence is encoded by a position embedding method, so that the encoded token sequence carries the information of the relative position of each token in the image. The encoded token sequence is padded to the preset maximum token length, and the padded token sequence is the feature map obtained by preprocessing.

[0023] Preferably, the specific method of using the noise predictor to predict the noise added in the forward noisy process is: the feature maps of the structural image and the true dose distribution map at time t in the forward noisy process are fused by element-by-element addition to obtain the predicted noise at time t.

[0024] Preferably, in step 4), the improved FiT model performs forward noise addition on the feature map of the real dose distribution map under the guidance of the structural image, specifically:

[0025]

[0026] Among them, x t-1 、x t is the characteristic graph of the real dose distribution graph with noise at time t-1 and time t in the forward noise process, x0 is the characteristic graph of the real dose distribution graph without noise, z1, z t is the noise added at the first and tth moments in the forward noise addition process. All added noises obey the normal distribution. t are the preset hyperparameters.

[0027] Preferably, the objective function of improving the FiT model pre-training in step 4) is:

[0028]

[0029] Among them, θ represents the trainable parameters in the conditional model, represents the trainable parameters in the optimization conditional model to minimize the function value, ε is the actual added noise at time t in the forward noise addition process, and ε t,θ is the noise added at time t predicted by the noise predictor, ‖‖ is the Euclidean distance of the vector, E (x,y) represents the expectation under the Euclidean distance, x is the structural image input to the conditional model, and y is the true dose distribution map.

[0030] Preferably, the acquiring of the noise image refers to acquiring the noise image by sampling from a Gaussian distribution.

[0031] Preferably, the improved FiT model performs reverse denoising on the feature map of the noise image under the guidance of the structure image to be predicted, specifically:

[0032]

[0033] Among them, y′ t-1 ,y′ t ~N(0,1) is the feature map of the noise image at time t-1 and time t during the reverse denoising process, α t , β t is the hyperparameter at time t, ε t,θ The noise added at time t to improve the noise predictor prediction during the pre-training process of the FiT model, θ represents the trainable parameters in the conditional model, δ t is the noise intensity removed cumulatively at time t during the reverse denoising process, and ε′ is a normal distribution.

[0034] On the other hand, the present invention also proposes an automatic prediction system for dose distribution in radiotherapy plan based on improved FiT, which is used to implement the above automatic prediction method.

[0035] Compared with the prior art, the present invention has the following beneficial technical effects:

[0036] 1. The present invention uses a diffusion model to automatically predict the radiation dose distribution of a cancer patient's treatment plan, thereby reducing the burden on dosimetry technicians and speeding up the radiotherapy procedure.

[0037] 2. The present invention replaces the traditional UNet architecture with the Flexible Vision transformer architecture. Compared with the DiT architecture, the FiT architecture improves the model generation capability, solves multi-resolution training and reasoning capabilities, and improves the accuracy of the dose distribution prediction map.

[0038] 3. The present invention solves the problem of difficulty in retaining basic structural information and unstable results by introducing structural images (CT images, PTV and OARs), thereby generating a more accurate dose map. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A block diagram of a method for automatically predicting dose distribution in a radiotherapy plan based on improved FiT provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of the forward noise addition process of the diffusion model provided in the embodiment of the present application;

[0041] Figure 3 A schematic diagram of a model of a feature extraction condition architecture provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of a patching process provided in an embodiment of the present application;

[0043] Figure 5 A schematic diagram of noise-adding training for the model forward process provided in an embodiment of the present application;

[0044] Figure 6 A schematic diagram of the inference process of the diffusion model inverse denoising provided in an embodiment of the present application;

[0045] Figure 7 Schematic diagram of the inverse denoising process of the diffusion model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0047] An embodiment of the present invention provides an automatic prediction method for dose distribution in radiotherapy plans based on improved FiT, which completes the radiation dose prediction in lung cancer treatment through a diffusion model. Compared with the traditional dose prediction method that relies on the professional knowledge and experience of radiologists, it reduces excessive dependence on radiologists and improves the efficiency of radiotherapy plan execution. The embodiment combines Transformer with the diffusion model to further improve the accuracy of dose prediction, and enhances the controllability of the prediction process by using a conditional architecture to extract semantic features of structural images.

[0048] The automatic prediction method of dose distribution in radiotherapy plan described in the present invention is implemented based on an improved FiT model, which is a diffusion model using Flexible Vision transformer architecture, which is different from the traditional diffusion model based on U-Net architecture. The improved FiT model of the present invention is composed of the traditional FiT (Flexible Vision transformer) model as the base model and the conditional model of the present invention, and utilizes the advanced characteristics of the FiT architecture to improve the accuracy and efficiency of radiation dose prediction in cancer treatment.

[0049] like Figure 1 As shown, the automatic prediction method of dose distribution in radiotherapy plan based on improved FiT provided by the embodiment includes the following steps:

[0050] S1, obtaining a structural image containing information about the patient's lesion location and a true dose distribution map corresponding to the structural image as a training data set.

[0051] The structural image x containing the patient's lesion location information includes the patient's CT image, planning target volume (PTV) and segmentation masks of organs at risk (OARs). In this embodiment, the specific implementation purpose is to predict the dose distribution of radioactive therapeutic drugs in the optimal radiotherapy plan for lung cancer patients. The structural image x includes a CT image of the lungs of lung cancer patients, a planning target volume and a segmentation mask of organs at risk. The planning target volume is a regional image artificially divided in the CT image, and the segmentation mask of the organs at risk is also obtained by segmenting the CT image. The real dose distribution map is a dose distribution map of radioactive therapeutic drugs in the patient's body marked on the basis of the CT image.

[0052] Wherein, the size of the structural image x is (2+O)*H*W, 2 represents the CT image and PTV, O represents the number of OARs, and H and W define the length and width of the image. The present invention utilizes structural image data training to improve the ability of the FiT model to generate a predicted dose distribution map, and the generated predicted dose distribution map can maximize the therapeutic effect of radioactive drugs on the target planning volume while minimizing potential damage to dangerous organs.

[0053] The present invention introduces the FiT architecture, so the size of the structural image and the real dose distribution map only needs to satisfy the image area HW≤256 2 .

[0054] S2, constructing a conditional model, and mounting the conditional model on the base model to form an improved FiT model.

[0055] like Figure 3As shown, the present invention proposes an innovative architecture, called conditional architecture, as an architecture for conditional models. The architecture combines the advantages of the ControlNet structure with the flexibility of the FiT architecture to achieve more efficient model performance and more precise task control.

[0056] Specifically, the conditional model includes N FiT blocks and N+1 zero linear layers, wherein the first zero linear layer is used as the first layer of the conditional model and is serially connected to the N FiT blocks, and the output of each FiT block is processed by a zero linear layer and then output to the base model. The specific structure of the FiT block is common knowledge in the art and is not expanded in the present invention.

[0057] In order to achieve the integration of ControlNet structure and FiT architecture, the present invention selectively applies the FiTBlock structure in the FiT architecture to the initial N basic blocks of the ControlNet structure. This selective integration allows the model to introduce the control capability of specific tasks through the ControlNet structure while maintaining the processing capability of the FiT base architecture. During training, we freeze the parameters of the FiT base model and only train the ControlNet part, so that the base model has the ability to predict dose distribution.

[0058] The base model adopts the traditional FiT (Flexible Vision transformer) model. The traditional FiT model is a diffusion model based on Transformer, which uses the diffusion process to gradually improve the feature representation of the token sequence. The base model includes multiple FiT blocks. The structure of the FiT block is well known in the art and is not described in detail in the present invention. In one embodiment of the present invention, N trainable FiT blocks are generated with the first N FiT blocks of the base model. Each trainable FiT block is trained independently and can be optimized according to specific task requirements. The outputs of these trainable FiT blocks are the key to model learning and adaptation because they allow the model to switch and adjust between different tasks.

[0059] The structural image x is input to the first zero linear layer in the conditional model, and the output of the zero linear layer is combined with the input of the base model as the input of the first FiT block in the conditional model. The output of the i-th FiT block in the conditional model is connected to a zero linear layer. The output processed by the zero linear layer is added to the output of the i-th FiT block of the corresponding base model to form a combined output. Subsequently, the combined output is used as the input of the i+1th FiT block of the subsequent base model. This design allows the model to utilize the control information provided by the ControlNet structure at each processing stage while maintaining the efficient data processing capabilities of the FiT architecture.

[0060] In one embodiment of the present invention, N=13 is used as the final model parameter.

[0061] S3, performing data preprocessing on the structural image and the true dose distribution map, and processing the image into a long sequence low-dimensional feature map.

[0062] S3.1 In the present invention, the encoder part of the variational autoencoder (VAE) (hereinafter referred to as VAE encoder) is used to achieve image compression.

[0063] In one embodiment of the present invention, specifically, the VAE encoder receives an image to be preprocessed, the original size of the image to be preprocessed is 512×512×3, and compresses it into a low-dimensional latent space representation z. The size of this latent space is 64×64×4, where the image size I is equal to 64 and the number of channels C is equal to 4.

[0064] The output z of S3.2 VAE encoder is further processed by a process called "Patchify". Patching is a method of dividing an image into small non-overlapping regions (or "patches") according to a specified patch size, which can then be processed or analyzed independently. In the present invention, the patches obtained by patching the spatial representation z are called tokens, and each token represents a patch in the image.

[0065] like Figure 4 As shown in Figure 1, in order to ensure that the input data can be processed efficiently later, the present invention defines precise input specifications. Specifically, given a spatial representation of shape I×I×C (the noisy latent image from VAE), it is first “patched” into The patches are rearranged in sequence on the same dimension to obtain a token sequence of length T, where the hidden dimension is d.

[0066] In this embodiment, the value of the patch size p is set to 4. In this way, the compressed 64×64 low-dimensional feature image is divided into non-overlapping 16×16 patches, each of which contains a 4×4 pixel area in the original image, and the dimension of each patch is 4 (corresponding to the number of color channels). These patches are rearranged and transformed to obtain a token sequence of length T=256.

[0067] S3.3 The patched token sequence is then input into the first layer of the traditional Flexible Vision Transformer (FiT) architecture. This step only uses the first layer of the FiT architecture, which is responsible for linearly embedding the input token sequence, and each embedded token has a dimension of d.

[0068] To further enhance the model's understanding of spatial structure, we use a frequency-based position embedding method, the sine-cosine version, to encode all input tokens. This position embedding method exploits the periodicity of sine and cosine functions and provides the model with fine-grained information about the relative position of tokens in the image.

[0069] S3.4 Figure 7 As shown in FIG. 1 , the length of the token sequence input to the improved FiT model must be less than the maximum token length L. In this embodiment, the maximum token length is defined as L=512. For the token sequence after position embedding, it is also necessary to pad it to the maximum token length for packaging. When calculating the loss function during model training, only the loss function of the denoised output token is calculated, and all padding tokens are discarded.

[0070] S4, constructing a noise predictor model, wherein the noise predictor is used to predict the noise added in the noise adding process.

[0071] The noise predictor model is a key component used in the present invention to predict and process noise. It predicts the noise added by the improved FiT model at each moment in the forward noise addition process through formula reasoning.

[0072] The specific method of the noise predictor for predicting noise is: the feature map of the structural image and the real dose distribution map at time t in the forward noise addition process is fused by element-by-element addition to obtain the predicted noise at time t, thereby allowing the structural information to be transmitted in the noise predictor.

[0073] The formula is as follows:

[0074] ε t,θ =f(g(x),g(y t ),γ t )

[0075] Among them, ε t,θ is the noise added by the improved FiT model predicted by the noise predictor at time t in the forward noise addition process, θ represents the trainable parameters of the conditional model, g(x) is the feature map after structural image preprocessing, and g(y t ) is the characteristic diagram of the real dose distribution diagram at time t in the forward noise addition process, γ t is the accumulated noise intensity at time t during the forward noise addition process.

[0076] S5: Input the preprocessed feature map of the structural image and the feature map of the true dose distribution map into the improved FiT model to train the improved FiT model.

[0077] The training optimization is performed by updating the parameters of the conditional model so that the noise predicted in the reverse process is as equal as possible to the noise added in the forward noisy process. The base model is frozen during the training process, and the noise predictor is also not involved in the parameter update.

[0078] For this purpose, the objective function is defined as follows:

[0079]

[0080] Among them, θ represents the trainable parameters in the conditional model, represents the trainable parameters in the optimization conditional model to minimize the function value, ε is the actual added noise at time t in the forward noise addition process, and ε t，θ is the noise added at time t predicted by the noise predictor, |||| is the Euclidean distance of the vector, E (x，y) represents the expectation under the Euclidean distance, x is the structural image input to the conditional model, and y is the true dose distribution map.

[0081] The model forward process noise training process is as follows Figure 5 As shown, the preprocessed feature maps of the structural image and the true dose distribution map are input into the improved FiT model of the present invention. Specifically, the feature map of the structural image is input into the conditional model in the improved FiT model, and the feature map of the true dose distribution map is input into the base model in the improved FiT model. The improved FiT model performs forward noise addition on the feature map of the true dose distribution map under the guidance of the structural image, and the noise predictor predicts the noise added in the forward noise addition process. The forward noise addition process of the improved FiT model is the process of gradually applying noise to the real data. Assuming the true dose distribution map x0~q(x) in the embodiment, the forward process is to continuously add noise to the image until the image becomes a pure noise, such as Figure 2 As shown in the figure, the real dose distribution diagram containing noise at each moment is defined as x1, x2, ..., x T , the specific formula is as follows:

[0082]

[0083] Among them, q(x t |x t-1 ) is the conditional probability of generating the t-th feature map from the t-1th feature map, x t-1 、x t is the characteristic diagram of the real dose distribution diagram with noise at time t-1 and time t in the forward noise addition process, is the hyperparameter at time t, I is the unit matrix, N(·) represents Gaussian distribution, q(x 1：T |x0) is the conditional probability of generating the Tth feature map from the initial unnoised true dose distribution feature map, T is the number of forward denoising iterations, It represents the multiplication from time t=1 to time T.

[0084] In the process of adding noise, it is necessary to use the reparameterization technique to calculate the x at each moment. t To take a sample:

[0085] Assume α t =1-β t ,and Expand x t You can get:

[0086]

[0087] Among them, z i is noise, which obeys normal distribution, z1, z2, ... ~ N (0, I), α t and β t is the hyperparameter preset in the sampling process, x0 refers to the feature map at time 0, x t It is the feature map at time t after adding noise.

[0088] If you want to randomly sample a sample of a Gaussian distribution from a certain distribution, this process cannot backpropagate the gradient. Therefore, it is necessary to make it differentiable through the above-mentioned reparameterization technique so that the gradient can be backpropagated.

[0089] Due to the additivity of independent Gaussian distributions, that is, so

[0090] So we can mix two Gaussian distributions to get a standard deviation of A mixture of Gaussian distributions.

[0091] Furthermore, the present invention predicts the cumulative noise intensity γ t , improve the hyperparameter α of the FiT model t and β t All regressions were performed. The hyperparameter α t The setting allows the model to adjust the scale of the feature map to adapt to the feature representation requirements of different levels.

[0092] In the process of model forward noise addition, as t increases, x t As T approaches infinity, x T is a complete Gaussian noise. In this embodiment, T = 1000, after 1000 times of adding noise, x T It can be regarded as pure Gaussian noise.

[0093] S6, using the trained improved FiT model, the dose distribution is predicted based on the structure image to be predicted.

[0094] like Figure 6 and Figure 7 As shown, this step utilizes the reverse denoising process of the model. Specifically, a noise image is first obtained. The feature maps of the noise image and the structure image to be predicted are preprocessed and respectively input into the base model and conditional model in the trained improved FiT model. Under the guidance of the structure image to be predicted, the improved FiT model reversely denoises the feature map of the noise image according to the noise predicted by the noise predictor, and gradually denoises the noise image until an image is generated. The generated image is the feature map of the predicted dose distribution map. After completing K iterations of the denoising process, the denoising mark is reconstructed and decoded according to the position information embedded in the preprocessing process to obtain the final generated image.

[0095] In this embodiment, the noise image is obtained by sampling noise from a Gaussian distribution as the noise image; the structural image x to be predicted includes a CT image of a lung cancer patient, a planned target volume, and a segmentation mask of an organ at risk; and the number of denoising iterations K is set to 30.

[0096] For this purpose, the inverse denoising function is defined as follows:

[0097]

[0098] Among them, y′ t-1 , y′ t ~N(0,1) is the feature map of the noise image at time t-1 and time t during the reverse denoising process, α t , β t is the hyperparameter at time t, ε t，θ is the noise added to the prediction of the noise predictor at time t, θ represents the trainable parameters in the conditional model, δ t is the noise intensity accumulated at time t in the reverse denoising process, and ε′~N(0,I) is normally distributed. In this example, the hyperparameter α t Set to 0.98, β t Set to 0.02. The above formula only defines a denoising function for one iteration from time t to time t-1. After K iterations, the feature map of the predicted dose distribution image is obtained.

[0099] After denoising for K times from time t = K, the denoised feature map at time t = 0 is obtained in the form of a token sequence with a token length of T = 256. The token sequence is rearranged to transform the size of the latent space to 64 × 64 × 4, where the image size is equal to 64 and the number of channels C is equal to 4. The VAE decoder receives a latent space image size to restore the image. The restored image size is 512 × 512 × 3, which is the final predicted dose distribution map.

[0100] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An automatic prediction method for dose distribution in radiotherapy planning based on improved FiT, characterized in that: include: Acquire a noise image and a structural image to be predicted containing information about the patient's lesion location, and preprocess the noise image and the structural image to be predicted to obtain feature maps of the noise image and the structural image to be predicted; Inputting the feature maps of the noise image and the structure image to be predicted into the pre-trained improved FiT model, the improved FiT model performs reverse denoising on the feature map of the noise image under the guidance of the structure image to be predicted, and obtains the feature map of the predicted dose distribution map; the improved FiT model includes a base model and a conditional model mounted on the base model, the base model is a pre-trained FiT model, receives the input noise image, and the conditional model receives the structure image to be predicted; The feature map of the predicted dose distribution map is reconstructed and decoded to obtain the predicted dose distribution map.

2. The method for automatically predicting dose distribution in radiotherapy planning based on improved FiT according to claim 1, characterized in that: The pre-training process of the improved FiT model is specifically as follows: 1) obtaining a structural image containing information about the patient's lesion location and a real dose distribution map corresponding to the structural image, wherein the structural image containing information about the patient's lesion location includes a CT image of the patient, a planned target volume, and a segmentation mask of an organ at risk; 2) Constructing a conditional model, using the pre-trained FiT model as a base model, and mounting the conditional model on the base model to form an improved FiT model; 3) Preprocessing the structural image and the true dose distribution map to obtain characteristic maps of the structural image and the dose distribution map respectively; 4) Freezing the base model, inputting the feature map of the structural image into the conditional model, inputting the feature map of the real dose distribution map into the base model, training the improved FiT model, during the training process, the improved FiT model performs forward noise addition on the feature map of the real dose distribution map under the guidance of the structural image, and using a noise predictor to predict the noise added in the forward noise addition process; The training objective is to minimize the difference between the noise predicted by the noise predictor and the noise actually added by the improved FiT model.

3. The method for automatically predicting dose distribution in radiotherapy planning based on improved FiT according to claim 1 or 2, characterized in that: The conditional model includes N FiT blocks and N+1 zero linear layers, wherein the first zero linear layer is used as the first layer of the conditional model and is serially connected to the N FiT blocks, and the output of each FiT block is processed by a zero linear layer and then output to the base model; The mounting relationship between the base model and the conditional model is specifically as follows: The output of the first zero linear layer in the conditional model is added and combined with the input of the base model as the input of the first FiT block in the conditional model. The output of the i-th FiT block in the conditional model is processed by the zero linear layer and added and combined with the output of the i-th FiT block in the base model as the input of the i+1-th FiT block in the base model. The number of FiT blocks in the conditional model is less than the number of FiT blocks in the base model.

4. The method for automatically predicting dose distribution in radiotherapy planning based on improved FiT according to claim 1 or 2, characterized in that: The preprocessing method of the noise image and the structure image to be predicted is consistent with the preprocessing method of the structure image and the true dose distribution map in the pre-training process of the improved FiT model. The specific preprocessing method is: For the image to be preprocessed, the encoder part of the variational autoencoder is compressed into a low-dimensional latent space representation. The low-dimensional latent space representation is first patched into multiple tokens of the same size and non-overlapping. Multiple tokens are rearranged into a token sequence, and then the token sequence is encoded by a position embedding method, so that the encoded token sequence carries the information of the relative position of each token in the image. The encoded token sequence is padded to the preset maximum token length, and the padded token sequence is the feature map obtained by preprocessing.

5. The method for automatically predicting dose distribution in radiotherapy planning based on improved FiT according to claim 1 or 2, characterized in that: The specific method of using the noise predictor to predict the noise added in the forward noisy process is: the feature maps of the structural image and the true dose distribution map at time t in the forward noisy process are fused by element-by-element addition to obtain the predicted noise at time t.

6. The method for automatically predicting dose distribution in radiotherapy planning based on improved FiT according to claim 2, characterized in that: In step 4), the improved FiT model performs forward noise addition on the feature map of the real dose distribution map under the guidance of the structural image, specifically: Among them, x t-1 、x t is the characteristic graph of the real dose distribution graph with noise at time t-1 and time t in the forward noise process, x0 is the characteristic graph of the real dose distribution graph without noise, z1, z t is the noise added at the first and tth moments in the forward noise addition process. All added noises obey the normal distribution. t are the preset hyperparameters.

7. The method for automatically predicting dose distribution in radiotherapy planning based on improved FiT according to claim 2, characterized in that: The objective function of improving the FiT model pre-training in step 4) is: Among them, θ represents the trainable parameters in the conditional model, represents the trainable parameters in the optimization conditional model to minimize the function value, ε is the actual added noise at time t in the forward noise addition process, and ε t,θ is the noise added at time t predicted by the noise predictor, ‖‖ is the Euclidean distance of the vector, E (x,y) represents the expectation under the Euclidean distance, x is the structural image input to the conditional model, and y is the true dose distribution map.

8. The method for automatically predicting dose distribution in radiotherapy planning based on improved FiT according to claim 1, characterized in that: The acquiring of the noise image refers to acquiring the noise image by sampling from a Gaussian distribution.

9. The method for automatically predicting dose distribution in radiotherapy planning based on improved FiT according to claim 1 or 2, characterized in that: The improved FiT model performs reverse denoising on the feature map of the noise image under the guidance of the structure image to be predicted, specifically: Among them, y t ′ -1 ,y t ′ ~N(0,1) is the feature map of the noise image at time t-1 and time t during the reverse denoising process, α t , β t is the hyperparameter at time t, ε t,θ The noise added at time t to improve the noise predictor prediction during the pre-training process of the FiT model, θ represents the trainable parameters in the conditional model, δ t is the noise intensity removed cumulatively at time t during the reverse denoising process, ε ′ is a normal distribution.

10. An automatic prediction system for dose distribution in radiotherapy planning based on improved FiT, characterized in that: The system is used to implement the automatic prediction method according to claim 2, and the system comprises: A training data acquisition module, used to obtain structural images and true dose distribution maps; A model building module, used to build a conditional model and mount the conditional model on a base model to form an improved FiT model; A noise predictor module, used for constructing a noise predictor and predicting noise using the noise predictor; Model training module, used to train and improve the FiT model; The automatic prediction module is used to obtain the predicted dose distribution map through the trained improved FiT model.

Images