A method, system, and terminal for predicting tumor radiotherapy response based on variational autoencoders.

Abstract

This invention discloses a method, system, and terminal for predicting tumor radiotherapy response based on a variational autoencoder (VAE). The prediction method includes steps such as image acquisition and slicing, image fusion and preprocessing, construction of a modified VAE, pre-training the modified VAE, and prediction of radiotherapy response label values. In the modified VAE, convolutional layers replace fully connected layers in the encoder and decoder. This invention introduces masking techniques and the KL cost annealing method to improve the feature extraction capability of the encoder; it fuses two-dimensional PETpre images and two-dimensional Dose images, and the fused image is pre-trained using a modified VAE to obtain latent variables. These latent variables serve as input variables for the prediction model, and the SUV change value serves as the label value, predicting the change of SUV during radiotherapy, thus solving the problems of existing models having single input variables, poor encoder feature extraction capability, and low prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more specifically, to a method, system, and terminal for predicting tumor radiotherapy response based on a variational autoencoder. Background Technology

[0002] PET / CT is a nuclear medicine imaging technique that combines positron emission tomography (PET) with computed tomography (CT). Radiomics, using PET / CT images as analytical tools, is increasingly used in precision radiotherapy decision-making and personalized treatment for tumors. Radiomics and high-dimensional extraction of image features both have the potential to analyze non-invasive, phenotypic tumors and predict regional radiotherapy responses. The Standard Uptake Value (SUV) is defined as the tissue concentration of the tracer measured by the PET scanner divided by the injected activity. The SUV value is calculated using a formula that considers the amount of radionuclide injected into the patient, the patient's weight, and the amount of radionuclide accumulated in a specific region of the body. SUV is typically represented by pixel or voxel intensity values ​​in the Region of Interest (ROI) of the image, which are then converted into activity concentration. Mid-stage SUV values ​​are a good indicator of and predictive of radiotherapy efficacy.

[0003] With the deepening research into artificial intelligence and radiomics in the field of precision radiotherapy, researchers both domestically and internationally are gradually applying traditional machine learning and deep learning to predict radiotherapy responses. Variational autoencoders (VAEs), proposed by Kingma et al. in 2014, have become an important unsupervised generative model, and VAEs have become one of the most popular methods for unsupervised learning of complex distributions. VAEs provide a probabilistic way to describe observations in a latent space. Therefore, instead of constructing an encoder that outputs a single value to describe each latent state attribute, it formulates the encoder to describe the probability distribution of each latent attribute. It can learn the mapping from the latent encoding space to the data generation space. Patient images are fed as input into the VAE encoder, which generates a latent variable containing features of the original image. This latent variable is rich in features of the original image. The latent variable is then used directly through the VAE decoder to generate another image, and a regression prediction model is constructed by combining the latent variable with other regression models.

[0004] However, existing models suffer from several drawbacks. Firstly, they can only accept a single variable X and label Y as input, failing to extract features from both images simultaneously. Secondly, while variational autoencoders (VAEs) typically improve both the encoder's image feature extraction and the decoder's image reconstruction capabilities by designing a minimum loss function, with nearly identical improvements, if only the encoder's feature extraction capability is needed, VAEs should prioritize this capability rather than simultaneously training a high-performing decoder. Thirdly, existing models rarely predict the SUV (Optical Vehicle Value) change in tumor radiotherapy response, and a model with high accuracy in predicting SUV change based on VAEs has not yet been developed.

[0005] Therefore, developing a tumor radiotherapy response prediction method and system with high prediction accuracy based on improved VAE and SUV change values ​​is an urgent technical problem to be solved. Summary of the Invention

[0006] Due to the aforementioned deficiencies in existing technologies, this invention provides a tumor radiotherapy response prediction method, system, and terminal based on variational autoencoders. It fuses two-dimensional PETpre images and two-dimensional Dose images, and obtains latent variables from the fused images through modified VAE pre-training. These latent variables serve as input variables for the prediction model, and the SUV change value serves as the label value. This predicts the change in SUV during mid-radiotherapy, thus addressing the problems of existing models having single input variables, poor encoder feature extraction capabilities, and low prediction accuracy.

[0007] To achieve the above objectives, on the one hand, the present invention provides a method for predicting tumor radiotherapy response based on a variational autoencoder, characterized by comprising the following steps:

[0008] Step S1, Image Acquisition and Slicing: Acquire three-dimensional PETpre and three-dimensional Dose images from the original dataset. Slice the three-dimensional PETpre and three-dimensional Dose images along the Z-axis based on the XY plane where the radiation dose points are located. The three-dimensional PETpre and three-dimensional Dose images are sliced ​​in the same slicing orientation to obtain two-dimensional PETpre and two-dimensional Dose images.

[0009] Step S2, Image Fusion and Preprocessing: The two-dimensional PETpre image and the two-dimensional Dose image are fused to obtain a fused image dataset. The fused image is then processed using masking techniques. The fused image dataset is divided into two parts: one part is used as a training set and the other part is used as a test set. The slice images of the same patient only appear in one of the training or test datasets. On the one hand, this prevents data leakage. On the other hand, if the slice images of the same patient appear in both the training and test sets, the model is prone to overfitting.

[0010] Step S3: Construct an improved variational autoencoder (improved VAE): Replace the fully connected layers in the encoder and decoder of the VAE with convolutional layers, and add a batch normalization layer, a LeakyReLU layer and a Dropout layer after each convolutional layer.

[0011] Step S4: Pre-train the improved VAE: Use the training set and test set to reconstruct images from the encoder and decoder of the improved VAE; introduce the Kullback-Leible (KL) cost annealing method to calculate the loss value of the improved VAE model.

[0012] Step S5: Predicting changes in SUV during radiotherapy: Connect the encoder and hidden layer of the pre-trained variational autoencoder to a multilayer perceptron (MLP) to form a prediction model; import the fused image into the encoder and hidden layer for dimensionality reduction, and calculate the latent variable Z in the hidden layer using the formula z = μ + ε × σ, where ε is sampled from the standard normal distribution, and μ and σ are the mean and standard deviation of the distribution calculated by the modified VAE; input the latent variable Z and the change in tumor standard uptake value as variables X and Y, respectively, into the prediction model; the last fully connected layer in the MLP has an output dimension of 1, which can predict changes in SUV during radiotherapy.

[0013] An image dataset fused from two-dimensional PETpre and two-dimensional Dose images was used, and the KL costannealing method was introduced to pre-train the improved VAE to obtain a latent variable Z containing a large number of features from the two original images. The latent variable was used as the input variable of the MLP and the SUV change value was used as the label value. The MLP was then used to predict the change of SUV during the mid-radiotherapy period.

[0014] Furthermore, in step S2, an iterative attention feature fusion method is used to fuse the two-dimensional PETpre image and the two-dimensional Dose image to obtain a fused image dataset. The iterative attention feature fusion method has higher recognition efficiency compared to ordinary addition-subtraction fusion.

[0015] Furthermore, in step S2, the number of images in the training set and the test set are similar. This feature ensures that the limited dataset can guarantee that the VAE is trained sufficiently and that the prediction module has a sufficient dataset for validation. However, it does not preclude the choice of other allocation ratios.

[0016] Further, in step S3, the first, second, and third convolutional layers of the encoder have 3*3 kernels, the fourth convolutional layer has 4*4 kernels, and the stride is 2*2 for all layers; the first transposed convolutional layer of the decoder has 4*4 kernels, the second, third, and fourth transposed convolutional layers have 3*3 kernels, and the stride is 2*2 for all layers; after the four convolutional layers of the encoder and after the three transposed convolutional layers of the decoder (excluding the last transposed convolutional layer), a batch normalization layer is added; the activation layer uses the LeakyReLU function, and the calculation formula of the LeakyReLU function is: LeakyReLU=max(0,x)+α*min(0,x), where α is a constant less than 1; a Dropout layer is added after the LeakyReLU layer, and a Sigmoid layer is added as the activation function after the last transposed convolutional layer in the decoder. The Sigmoid function outputs in the range of (0, 1), and can be represented as a probability value, enabling better prediction of pixel values.

[0017] Further, in step S4, the loss function for pre-training the improved variational autoencoder is: loss = ω * KL loss +l Recon Where ω is a variable weighting parameter—the growth coefficient, KL loss Let l be the KL divergence loss value. Recon The image reconstruction loss value is defined by the following formula: The formula for calculating the KL divergence loss value is as follows: MSE loss This represents the mean squared error of the reconstruction loss between the original image and the reconstructed image, where x is the original image. To reconstruct the image, σ 2 and μ 2 The mean and standard deviation of the distribution calculated for the improved VAE.

[0018] Furthermore, the growth coefficient ranges from 0.002 to 0.05, and the prediction effect is better within this range.

[0019] Furthermore, the block size used in the masking technique is 1 or 2; when the block size is 1, the mask ratio used in the masking technique ranges from 0.16 to 0.84, and the prediction effect is better within this range; when the block size is 2, the mask ratio used in the masking technique ranges from 0.16 to 0.86, and the prediction effect is better within this range.

[0020] Furthermore, in step S5, the multilayer perceptron consists of three fully connected layers, and ReLU activation function layers are added between the fully connected layers to further improve model performance.

[0021] On the other hand, the present invention provides a tumor radiotherapy response prediction system based on a variational autoencoder, characterized in that it includes:

[0022] The image preprocessing module is used to perform the same slicing process on the acquired 3D PETpre image and 3D Dose image in the same slice orientation to obtain 2D PETpre image and 2D Dose image, and to perform image fusion on the 2D PETpre image and 2D Dose image to obtain a fused image dataset, and then to perform masking processing on the fused image.

[0023] An improved variational autoencoder module includes an encoder and decoder where convolutional layers replace fully connected layers; each convolutional layer is followed by a batch normalization layer, a LeakyReLU layer, and a Dropout layer; the KL cost annealing method is used to calculate the loss value of the improved variational autoencoder model.

[0024] A training module is used to pre-train the improved variational autoencoder module;

[0025] The prediction module includes the encoder and hidden layer of the pre-trained variational autoencoder and a multilayer perceptron. After the image is dimensionality reduced by the encoder, the resulting latent variable Z contains a large number of features of the original image. Z and the tumor standard uptake value are fed into the multilayer perceptron for processing to form the prediction model.

[0026] In another aspect, the present invention provides a tumor radiotherapy response prediction terminal based on variational autoencoder, characterized in that it includes: one or more memories and one or more processors;

[0027] The one or more memories are used to store computer programs;

[0028] The one or more processors are connected to the memory and are used to run the computer program to perform the above-described method for predicting tumor radiotherapy response.

[0029] Compared with the prior art, the above invention has the following advantages or beneficial effects:

[0030] (1) This invention uses convolutional layers instead of the fully connected layers of traditional VAEs, which can better process image features;

[0031] (2) This invention cleverly combines three different network models: iterative attention feature fusion, VAE and MLP, so that the features of the two original images are considered as analysis variables at the same time, thereby improving the prediction accuracy and adaptability of the prediction model.

[0032] (3) The method of the present invention uses two methods, masking technology and KL cost annealing, to improve the encoder’s ability to extract image features. At the same time, by adjusting model parameters such as masking ratio, block size, and growth coefficient, the accuracy of the system’s prediction of SUV change value in the mid-term radiotherapy of unknown patients is further improved. Attached Figure Description

[0033] The invention, its features and advantages will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0034] Figure 1 This is a flowchart of the steps in a tumor radiotherapy response prediction method according to an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of a tumor radiotherapy response prediction method in one embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of a masking technique in one embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of the network architecture of the prediction model in one embodiment of the present invention;

[0038] Among them, 1. Hidden layer; 2. Multilayer perceptron. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but these are not intended to limit the scope of the invention.

[0040] The terms “comprising,” “including,” and any variations thereof are intended to cover non-exclusive inclusion, such as a process, method, system, product, or apparatus that includes a series of steps or units that is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0041] In the following detailed description, numerous specific details are set forth to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that well-known control algorithms are not shown in detail to avoid obscuring the gist of the invention; and that the masking techniques, KL cost annealing methods, iterative attention feature fusion, and MLP techniques involved in the following effect examples are readily available prior art.

[0042] Example 1

[0043] See Figure 1 and Figure 2 This embodiment provides a method for predicting tumor radiotherapy response based on variational autoencoders, including the following steps:

[0044] Step S1, Image Acquisition and Slicing: Acquire 3D PETpre and 3D Dose images from the original dataset. Using the XY plane containing the radiation dose points as the slicing plane, slice the 3D PETpre and 3D Dose images along the Z-axis. Perform the same slicing process on the 3D PETpre and 3D Dose images at the same slicing orientation to obtain 2D PETpre and 2D Dose images, and unify the image sizes. For example, the image size is uniformly set to 56 pixels * 56 pixels.

[0045] Step S2, Image Fusion and Preprocessing: The two-dimensional PETpre image and the two-dimensional Dose image are fused to obtain a fused image dataset. The fused image is then processed using masking technology. The fused image dataset is divided into two parts: one part is used as a training set and the other part is used as a test set. The slice images of the same patient only appear in one of the training set or the test set.

[0046] As a preferred technical solution, in step S2, the iterative attention feature fusion method is used to fuse the two-dimensional PETpre image and the two-dimensional Dose image to obtain a fused image dataset. In step S2, the number of images in the training set and the test set are similar, approximately 1:1, to better allocate the limited dataset, ensuring that the allocated dataset can both guarantee sufficient training for the VAE and sufficient data for validation of the prediction module, but it is not necessarily 1:1.

[0047] Masking techniques involve randomly masking the original image, forcing the encoder to infer the original image features of the masked block from the unmasked image data near the masked block. Implementation primarily involves two variable parameters: the mask ratio and the block size. The block size is the size of a single mask block, and the mask ratio is the ratio of the masked area to the original image area. See also... Figure 3The left image is the original image, and the right image is the masked image. The mask blocks (small black blocks) are randomly distributed on the image. As the mask ratio increases or the block size increases, the masked area will expand.

[0048] Step S3: Construct an improved VAE: Replace the fully connected layers in the encoder and decoder of the variational autoencoder with convolutional layers, and add a batch normalization layer, a LeakyReLU layer and a Dropout layer after each convolutional layer.

[0049] Figure 4 The convolutional and deconvolutional layers, along with the hidden layer 1 in the middle, constitute the architecture of the entire VAE. As a preferred technical solution, the first, second, and third convolutional layers of the encoder have 3*3 kernels, the fourth convolutional layer has 4*4 kernels, and the stride is 2*2 for all layers. The first transposed convolutional layer of the decoder has 4*4 kernels, the second, third, and fourth transposed convolutional layers have 3*3 kernels, and the stride is 2*2 for all layers. Batch normalization layers are added after the four convolutional layers of the encoder and after the three transposed convolutional layers of the decoder (excluding the last transposed convolutional layer). The activation layer uses the LeakyReLU function, which is calculated as: LeakyReLU = max(0, x) + α*min(0, x), where α is a constant less than 1. A Dropout layer is added after the LeakyReLU layer, and a Sigmoid layer is added after the last transposed convolutional layer in the decoder as the activation function. It should be noted that the figure only shows the convolutional layer and the fully connected layer, and does not show the batch normalization layer, activation layer and dropout layer.

[0050] Step S4: Pre-train the improved VAE: Use the training and test sets to reconstruct images from the encoder and decoder of the improved VAE; introduce the KL cost annealing method to calculate the loss value of the improved VAE model, which tends to improve the encoder's capabilities. The KL cost annealing method adds a weight parameter that changes with the number of training iterations before the KL divergence. Therefore, the loss function for pre-training the improved VAE is: loss = ω * KL loss +l Recon Where ω is a variable weighting parameter—the growth coefficient, KL loss Let l be the KL divergence loss value. Recon The image reconstruction loss value is defined by the following formula: The formula for calculating the KL divergence loss value is as follows: MSE lossThis represents the mean squared error of the reconstruction loss between the original image and the reconstructed image, where x is the original image. To reconstruct the image, σ 2 and μ 2 The mean and standard deviation of the distribution calculated for the improved VAE.

[0051] As an example, at the beginning of VAE training, there's no need to worry about model overfitting; simply set the weight parameters to 0. This way, the loss function only contains reconstruction loss, allowing the encoder to learn more features from the original image. The expression for the loss function at this point is: loss = ω * KL loss +l Recon Where ω is a variable weight parameter, KL loss Let l be the KL divergence loss value. Recon The image reconstruction loss is calculated as follows: During training, the weight parameters are incremented by a fixed value after each training iteration until the weight parameters equal 1. This fixed value is a variable parameter, i.e., the growth factor.

[0052] Step S5, Predicting changes in tumor standardized uptake values ​​during mid-radiotherapy: See Figure 4 The encoder and hidden layer of the pre-trained VAE are connected to a multilayer perceptron 2 to form a prediction model. The fused image is imported into the encoder and hidden layer for dimensionality reduction. The latent variable Z in the hidden layer is calculated using the formula z = μ + ε × σ, where ε is sampled from a standard normal distribution, and μ and σ are the mean and standard deviation of the distribution calculated by the modified VAE. The latent variable Z and the change in tumor standard uptake value are respectively used as variable X and label value Y, and input into the multilayer perceptron 2. The formula for calculating the label value Y is: (PETmid - PETpre) / PETpre, where PETpre is the SUV value in the pre-treatment stage and PETmid is the SUV value in the mid-treatment stage. The last fully connected layer in the multilayer perceptron 2 has an output dimension of 1, which can predict the change in SUV in the mid-radiotherapy stage.

[0053] As a preferred technical solution, see Figure 4 In step S5, the multilayer perceptron 2 consists of three fully connected layers, with ReLU activation function layers (not shown in the figure) added between the fully connected layers.

[0054] This embodiment employs a tumor radiotherapy response prediction system based on a variational autoencoder to implement the above prediction method. The prediction system includes:

[0055] The image preprocessing module is used to perform the same slicing process on the acquired 3D PETpre image and 3D Dose image in the same slice orientation to obtain 2D PETpre image and 2D Dose image, and to perform image fusion on the 2D PETpre image and 2D Dose image to obtain a fused image dataset, and then to perform masking processing on the fused image.

[0056] An improved variational autoencoder module includes an encoder and decoder where convolutional layers replace fully connected layers; each convolutional layer is followed by a batch normalization layer, a LeakyReLU layer, and a Dropout layer; the KL cost annealing method is used to calculate the loss value of the improved variational autoencoder model.

[0057] A training module is used to pre-train the improved variational autoencoder module;

[0058] The prediction module includes the encoder and hidden layer of the pre-trained variational autoencoder and a multilayer perceptron. After the image is dimensionality reduced by the encoder, the resulting latent variable Z contains a large number of features of the original image. Z and the tumor standard uptake value are fed into the multilayer perceptron for processing to form the prediction model.

[0059] The prediction system, after processing two types of three-dimensional tumor image data using the aforementioned prediction method, predicted the changes in SUV (ultra-invasive lateral velocity) during the pre- and mid-term stages of treatment. The prediction results are then analyzed and evaluated using mean absolute error (MSE), mean squared error (MSE), and root mean square error (RMSE), and the effects of different parameter values ​​on model performance are measured.

[0060] The leave-one-out method was used for model validation. The validation process was as follows: the patients in the VAE test set were divided into two groups, with a total number of patients N and a ratio of 1:N-1. The group consisting of N-1 patients was used as the training set for the MLP, and the remaining patient was used as the test set. N training and testing cycles were performed, meaning each patient was used as part of the test set for one training cycle. The evaluation metrics obtained from each training cycle were analyzed, yielding different parameter results for the two methods of improving encoder capabilities. Table 1 shows the impact of the KL cost annealing method on the results under different growth coefficients; Table 2 shows the impact of masking techniques on the results under different combinations of mask ratios and block sizes; Table 3 shows the impact of different parameter combinations on the results for the two methods. The mask ratio ranged from 0.1 to 0.9, with parameter values ​​taken at 0.02 intervals, resulting in 42 different parameters. The block size had two possible values: 1 and 2. The growth coefficient ranged from 0.001 to 0.05, with parameter values ​​taken at 0.001 intervals, resulting in 50 different parameters. The table is arranged from lowest to highest average RMSE, showing only the top ten results and their corresponding parameter values. Choosing better mask ratios, block sizes, and growth factors can reduce the average RMSE for predicting SUV changes to 0.142–0.145.

[0061] Table 1. The impact of different growth coefficients on the results.

[0062]

[0063]

[0064] Table 2. Effects of different mask ratios and block sizes on the results.

[0065]

[0066] Table 3. The impact of different mask ratios, block sizes, and growth factors on the results.

[0067]

[0068]

[0069] Comparative Example 1

[0070] This comparative example uses a prediction method similar to that of Example 1 to predict SUV change values ​​for the fused image dataset used in Example 1. However, unlike Example 1, this comparative example does not perform masking on the fused image data and does not introduce the KL cost annealing method during pre-training. Table 4 shows the results of model validation using the leave-one-out method. It can be seen that before introducing masking and the KL cost annealing method, the average RMSE for predicting SUV change values ​​is 0.158.

[0071] Table 4 shows the results of using neither masking nor the KL cost annealing method.

[0072]

[0073] Example 2

[0074] This embodiment uses a prediction method similar to that of Embodiment 1 to predict the SUV change value of the fused image dataset used in Embodiment 1. The difference from Embodiment 1 is that this embodiment uses five-fold cross-validation for model validation. The validation process of five-fold cross-validation is as follows: patients in the VAE test set are divided into five groups with roughly equal numbers of patients per group. In one training iteration, one group of patients is used as the test set, and the remaining four groups are used as the training set for model evaluation. A total of five training and testing iterations are performed, with different training and test sets for each iteration. The evaluation metrics obtained from each training iteration are analyzed, yielding different parameter results for the two methods of improving encoder capabilities. Table 5 shows the impact of using the KL costannealing method on the results under different growth coefficients; Table 6 shows the impact of using masking techniques on the results under different combinations of mask ratios and block sizes; and Table 7 shows the impact of using different parameter combinations of the two methods on the results. The mask ratio ranges from 0.1 to 0.9, with parameter values ​​taken every 0.02 increments, resulting in 42 different parameters. The block size has two possible values: 1 and 2. The growth factor ranges from 0.001 to 0.05, with parameter values ​​taken every 0.001 increments, resulting in 50 different parameters. The table is arranged from lowest to highest average RMSE, showing only the top ten results and their corresponding parameter values. Choosing optimal mask ratio, block size, and growth factor can reduce the average RMSE of the predicted SUV change value to 0.193–0.194.

[0075] Table 5. The impact of different growth coefficients on the results.

[0076]

[0077] Table 6. The impact of different mask ratios and block sizes on the results.

[0078]

[0079] Table 7. The impact of different mask ratios, block sizes, and growth factors on the results.

[0080]

[0081] Comparative Example 2

[0082] This comparative example uses a prediction method similar to that of Example 1 to predict SUV change values ​​for the fused image dataset used in Example 1. However, unlike Example 1, this comparative example does not perform masking on the fused image data, does not introduce the KL cost annealing method during pre-training, and uses five-fold cross-validation for model validation. Table 8 shows the results of model validation using five-fold cross-validation. It can be seen that before introducing masking and the KL cost annealing method, the average RMSE for predicting SUV change values ​​is 0.200.

[0083] Table 8 shows the results of using neither masking nor the KL cost annealing method.

[0084]

[0085] In summary, the structural results of leave-one-out and five-fold cross-validation show that the model of this invention can accurately predict the change in SUV value during radiotherapy. Furthermore, comparing the results with and without the masking technique and KL cost annealing method, it is evident that the model exhibits good feature extraction capabilities under the masking and KL cost annealing method. Combining the two methods with appropriate parameter combinations can further enhance the encoder's image feature extraction capabilities. Specifically, the preferred value range for the growth coefficient is 0.002 to 0.05. The block size used in the masking technique is 1 or 2; when the block size is 1, the preferred value range for the mask ratio is 0.16 to 0.84; when the block size is 2, the preferred value range for the mask ratio is 0.16 to 0.86.

[0086] Example 3

[0087] This embodiment provides a tumor radiotherapy response prediction terminal based on variational autoencoder, including: one or more memories and one or more processors; the one or more memories are used to store computer programs;

[0088] The one or more processors are connected to the memory and are used to run the computer program to perform the tumor radiotherapy response prediction method described in Embodiment 1.

[0089] Optionally, the memory may include, but is not limited to, high-speed random access memory and non-volatile memory. For example, one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices; the processor may include, but is not limited to, a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0090] Optionally, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It is understood that the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0091] Those skilled in the art should understand that variations can be implemented by combining existing technology with the above embodiments, which will not be elaborated here. Such variations do not affect the essence of the present invention, and will not be elaborated here either.

[0092] The preferred embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and the devices and structures not described in detail should be understood as being implemented in a conventional manner in the art. Any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention using the methods and techniques disclosed above, or modify them into equivalent embodiments with equivalent changes, without departing from the scope of the present invention. This does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the present invention's technical solutions still fall within the protection scope of the present invention.

[0093] Those skilled in the art will recognize that the units, i.e., algorithm steps, of the various examples described in connection with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. A method for predicting tumor radiotherapy response based on variational autoencoders, characterized in that, Includes the following steps: Step S1, Image Acquisition and Slicing: Acquire 3D PETpre and 3D Dose images from the original dataset, based on the locations where radiation doses are present. The plane is sliced, and the three-dimensional PETpre image and the three-dimensional Dose image are sliced ​​along the Z-axis; the three-dimensional PETpre image and the three-dimensional Dose image are sliced ​​in the same slicing orientation to obtain a two-dimensional PETpre image and a two-dimensional Dose image. Step S2, Image Fusion and Preprocessing: The two-dimensional PETpre image and the two-dimensional Dose image are fused to obtain a fused image dataset, and then the fused image is processed using masking technology; The fused image dataset is divided into two parts: one part is used as the training set and the other part is used as the test set. The slice images of the same patient only appear in one of the training set or the test set. Step S3: Construct an improved variational autoencoder: Replace the fully connected layers in the encoder and decoder of the variational autoencoder with convolutional layers, and add a batch normalization layer after each convolutional layer. Layers and layer; Step S4: Pre-train the improved variational autoencoder: Reconstruct the image using the encoder and decoder of the improved variational autoencoder using the training set and test set; Introduce the KL cost annealing method to calculate the loss value of the improved variational autoencoder. Step S5: Predict changes in tumor standard uptake values ​​during mid-radiotherapy: Connect the encoder and hidden layer of the pre-trained variational autoencoder to a multilayer perceptron to form a prediction model; import the fused image into the encoder and hidden layer for dimensionality reduction, and then extract the latent variables in the hidden layer. From the formula Calculated, where Sampled from the standard normal distribution. and The distribution mean and standard deviation are calculated for the improved variational autoencoder; the latent variables are... The change in tumor standard uptake value was used as a variable. and The data is fed into the prediction model; the last fully connected layer in the multilayer perceptron has an output dimension of 1, which predicts the change in SUV during radiotherapy.

2. The method for predicting tumor radiotherapy response based on variational autoencoder according to claim 1, characterized in that, In step S2, the two-dimensional PETpre image and the two-dimensional Dose image are fused using an iterative attention feature fusion method to obtain a fused image dataset.

3. The method for predicting tumor radiotherapy response based on variational autoencoder according to claim 1, characterized in that, In step S3, the first, second, and third convolutional layers of the encoder have 3x3 kernels, the fourth convolutional layer has a 4x4 kernel, and the stride is 2x2 for all layers. The first transposed convolutional layer of the decoder has a 4x4 kernel, the second, third, and fourth transposed convolutional layers have 3x3 kernels, and the stride is 2x2 for all layers. A batch normalization layer is added after the four convolutional layers of the encoder and after the remaining three transposed convolutional layers of the decoder (excluding the last transposed convolutional layer). The activation layer uses... The function, the The formula for calculating the function is: ,in It is a constant less than 1; Added after the layer A layer is added after the last transposed convolutional layer in the decoder. The layer is used as the activation function.

4. The method for predicting tumor radiotherapy response based on variational autoencoder according to claim 3, characterized in that, In step S4, the loss function of the improved variational autoencoder is pre-trained: ,in The variable weighting parameter is the growth coefficient. This represents the KL divergence loss value. The image reconstruction loss value is defined by the following formula: The formula for calculating the KL divergence loss value is as follows: ,in This represents the mean squared error value of the reconstruction loss between the original image and the reconstructed image. The original image. To reconstruct the image, and The distribution mean and standard deviation are calculated for the improved variational autoencoder.

5. The method for predicting tumor radiotherapy response based on a variational autoencoder according to claim 4, characterized in that, The growth coefficient ranges from 0.002 to 0.

05.

6. A method for predicting tumor radiotherapy response based on a variational autoencoder according to claim 1 or 5, characterized in that, The block size used in the masking technique is 1 or 2; when the block size is 1, the mask ratio used in the masking technique ranges from 0.16 to 0.84; when the block size is 2, the mask ratio used in the masking technique ranges from 0.16 to 0.

86.

7. The method for predicting tumor radiotherapy response based on variational autoencoder according to claim 1, characterized in that, In step S5, the multilayer perceptron consists of three fully connected layers, and activation functions are added between the fully connected layers. layer.

8. A tumor radiotherapy response prediction system based on a variational autoencoder, characterized in that, include: The image preprocessing module is used to perform the same slicing process on the acquired 3D PETpre image and 3D Dose image in the same slice orientation to obtain 2D PETpre image and 2D Dose image, and to perform image fusion on the 2D PETpre image and 2D Dose image to obtain a fused image dataset, and then to perform masking processing on the fused image. The improved variational autoencoder module includes an encoder and decoder where convolutional layers replace fully connected layers; a batch normalization layer is added after each of the convolutional layers. Layers and Layer; The KL cost annealing method is used to calculate the loss value of the improved variational autoencoder; A training module is used to pre-train the improved variational autoencoder module; The prediction module includes the encoder and hidden layer of a pre-trained variational autoencoder and a multilayer perceptron; after the image is dimensionality-reduced by the encoder, the latent variables are obtained. Contains a large number of features from the original image, The tumor standard uptake value is fed into a multilayer sensor for processing.

9. A tumor radiotherapy response prediction terminal based on a variational autoencoder, characterized in that, include: One or more memories and one or more processors; The one or more memories are used to store computer programs; The one or more processors are connected to the memory and are used to run the computer program to perform the method as described in any one of claims 1 to 7.

Images