Tumor radiotherapy reaction prediction method and model based on double-path network

Abstract

The application provides a tumor radiotherapy reaction prediction method and model based on a double-path network, which comprises the following steps: firstly, pre-processing source data, and dividing the pre-processed source data into a training set, a verification set and a test set; then, constructing a deep learning model based on a double-path network; training the deep learning model based on the training set, adjusting the model parameters of the deep learning model based on the verification set, and obtaining optimized model parameters and an optimized deep learning model; evaluating the performance of the optimized deep learning model based on the test set and the optimized model parameters, and obtaining a trained double-path network model; finally, inputting a tumor data set to be predicted into the trained double-path network model, and predicting the radiotherapy reaction of the tumor. The method provided by the application takes into account the medical interpretability and high prediction accuracy, can intelligently assist doctors in setting radiotherapy doses, and reduces the harm of radiotherapy to patients.

Description

Technical Field

[0001] The present application relates to the interdisciplinary field of medical deep learning, and in particular to a method and model for predicting tumor radiotherapy response based on a dual-pathway network. Background Art

[0002] In recent years, with the advancement of tumor treatment technology, tumor radiotherapy has gradually developed towards precision and personalization. In order to better predict the tumor's response to radiotherapy and thus provide patients with the most appropriate treatment plan, the role of imaging technology in tumor diagnosis and treatment has become increasingly significant. Through multimodal medical imaging, such as magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET), etc., doctors can obtain the biological information and spatial distribution characteristics of the tumor, thereby assisting in efficacy evaluation and personalized treatment plan design. However, due to the heterogeneity and complexity of tumors, how to effectively analyze imaging data remains a key challenge in the precision of radiotherapy.

[0003] Artificial intelligence, especially deep learning, has made significant progress in various fields in recent years, and has also brought new opportunities for accurate prediction in the field of tumor radiotherapy. Deep learning models are good at automatically extracting high-level features from large amounts of complex data, making them widely used in medical image analysis. Especially in radiomics, deep learning can effectively analyze the potential high-order features in patient images, extract information related to tumor biological heterogeneity and treatment response, and thus predict treatment outcomes. Currently, common deep learning models include Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Variational Autoencoder (VAE), etc. These models are used in medical image analysis for tasks such as tumor segmentation, image registration, and tumor progression prediction.

[0004] While some research has been conducted on accurately predicting tumor response to radiotherapy, there is still significant room for improvement in effective models and medical interpretability. Therefore, developing a deep learning-based tumor radiotherapy prediction method that can accurately predict changes in tumors before and after radiotherapy while also ensuring interpretability has become a pressing technical challenge. Summary of the Invention

[0005] The embodiment of the application provides a tumor radiotherapy reaction prediction method and model based on a double-path network, which can predict the reaction of a tumor before and after radiotherapy, thereby serving as a reference for a doctor when formulating a radiotherapy dose for a patient, reducing the harm of radiotherapy to the patient's body, and improving the work efficiency and quality of the doctor.

[0006] To solve the above technical problems, in a first aspect, the application provides a tumor radiotherapy reaction prediction method based on a double-path network, which comprises the following steps: first, pre-processing source data and dividing the pre-processed source data into a training set, a verification set and a test set; then, constructing a deep learning model based on a double-path network; next, training the deep learning model based on the training set, adjusting the model parameters of the deep learning model based on the verification set, obtaining optimized model parameters and an optimized deep learning model; then, evaluating the performance of the optimized deep learning model based on the test set and the optimized model parameters, obtaining a trained double-path network model; and finally, inputting a tumor data set to be predicted into the trained double-path network model to predict the radiotherapy reaction of the tumor.

[0007] In some exemplary embodiments, the pre-processing of the source data comprises the following steps: sequentially performing gray image processing, RGB channel fusion and data batch stacking on the source data; wherein the gray image processing comprises: obtaining tumor pre-radiotherapy image standard uptake value (SUV) prePET, mid-radiotherapy image SUV midPET and radiotherapy dose Dose from patient PET / CT image source data, and then slicing the three groups of original data along the z-axis of PET / CT scanning to convert them into two-dimensional matrices; the RGB channel fusion comprises: performing RGB channel fusion on the prePET and Dose slices, that is, taking prePET as R channel data and Dose as G channel data to generate an RGB image, denoted as prePET; taking midPET as a gray image color channel to generate a gray image, denoted as midPET; and the data batch stacking comprises: stacking the prepared prePET and midPET slice sets into several batches according to the Batchsize set by the deep learning, to prepare for subsequent input model training, and the Batchsize is the number of samples contained in each batch.

[0008] In some exemplary embodiments, the preprocessed source data is divided into a training set, a validation set, and a test set, including: according to the principle of leave-one-out cross-validation, first, the test set of each round of cross-validation is assigned to a test patient to ensure that all patients and their data sets can be used as test sets in a certain round of cross-validation to achieve full coverage; second, according to the degree of patient response to radiotherapy, the patients are divided into three categories: high response, medium response, and low response, and the same number of patients and their data sets are selected from each category to form the validation set of the model; finally, the remaining unselected patients and their data sets are used as the training set of the model.

[0009] In some exemplary embodiments, a deep learning model is constructed based on a dual-path network, including the following steps: constructing a network main body architecture of the dual-path network; the network main body architecture of the dual-path network includes multiple Dual Path Block modules and two classifier layers; the network main body of the dual-path network obtains an output value through forward propagation, and after the output is adaptively pooled, it is further processed using a convolution classifier and a fully connected layer to obtain a final predicted feature map, and the feature map is converted to calculate the loss value and related prediction evaluation indicators; based on the network main body architecture of the dual-path network, the training set and the test set are converted into structured image data suitable for the dual-path network input, and the structured image data is input into the network main body architecture of the dual-path network for training, and the optimization is achieved by adjusting the model parameters, and finally a deep learning model is obtained based on the lowest loss value obtained from the input verification set; before the structured image data is input into the network main body architecture of the dual-path network for training, it also includes: converting the structured image data into a tensor tensor, and preprocessing and format conversion of the tensor tensor as the input of the network main body architecture of the dual-path network.

[0010] In some exemplary embodiments, after constructing the network main architecture of the dual-path network, and before converting the training set and the test set into structured image data suitable for the dual-path network input based on the network main architecture of the dual-path network, and inputting the structured image data into the network main architecture of the dual-path network for training, it also includes: optimizing the network main architecture of the dual-path network to obtain a shallow dual-path network model; wherein the optimization processing includes: reducing the number of Dual Path Block modules in the dual-path network, constructing a shallow network structure, and reducing the number of channels in each module to reduce the dimension of the feature map; using a smaller initial convolution kernel to reduce the number of parameters; and reducing the width expansion factor bw factor to reduce the width of the network.

[0011] In some exemplary embodiments, the main network architecture of the dual-path network also includes a specially designed network output layer; the specially designed network output layer is achieved by customizing the output layer; the customization includes: adding a 1×1 convolution layer at the end of the network to perform feature mapping on each pixel point, thereby outputting a feature map with the same spatial resolution as the input image to ensure the accuracy of pixel-level prediction; subsequently, the convolution feature is converted into a specified number of output channels through a fully connected layer to obtain a specific prediction value for each pixel, and an adaptive pooling layer is introduced to adapt the output size to input images of different sizes; finally, the output result is converted into a target shape through a reshape operation to meet the task requirements of pixel-level prediction. This process ensures that the output result has a spatial resolution consistent with the input image, so as to achieve accurate pixel-level prediction.

[0012] In some exemplary embodiments, a deep learning model is trained based on a training set, and model parameters of the deep learning model are adjusted based on a validation set to obtain optimized model parameters and an optimized deep learning model, including: obtaining pixel-level prediction output of the model through forward propagation, and calculating a loss function, wherein the loss function is expressed as a mean square error, and its calculation formula is:

[0013]

[0014] Among them, MSE is the mean square error, m represents the number of pixels in the first dimension, n represents the number of pixels in the second dimension, and midPET i，j is the true value, is the model prediction value, and its calculation result is used for back propagation to update the model parameters;

[0015] After each training, the model performance is evaluated based on the validation set, and the goodness of fit of the model is measured by calculating the coefficient of determination. The coefficient of determination is expressed as R 2 It is expressed as follows:

[0016]

[0017] Among them, m represents the number of pixels in the first dimension, n represents the number of pixels in the second dimension, and midPET i，j is the true value of the data, To predict the value of the model, the hyperparameters of the model are adjusted by evaluation on the validation set. The hyperparameters include learning rate and regularization term, and finally the optimized model parameters and deep learning model are obtained, thereby improving the prediction accuracy of tumor radiotherapy response.

[0018] In some example embodiments, performance evaluation is performed on the optimized deep learning model based on the test set and the optimized model parameters, and a trained dual-path network model is obtained, including: using the same evaluation index as in the training and verification, the evaluation index including mean square error and determination coefficient, and performing the test once after each epoch is completed and the model parameters are updated, which does not affect the gradient of the model.

[0019] In some example embodiments, the radiotherapy response of the tumor is predicted by inputting the tumor data set to be predicted into the trained dual-path network model, including: preprocessing the tumor data set to be predicted, the preprocessing including operations of gray image processing, RGB channel fusion and data batch stacking, so that the dimension of the preprocessed tumor data set to be predicted is the same as that of the tumor data set in the training set, the validation set and the test set, and a preprocessed tumor data set is obtained; and inputting the preprocessed tumor data set into the trained dual-path network model to predict the radiotherapy response of the tumor.

[0020] In a second aspect, the embodiments of the present application further provide a tumor radiotherapy response prediction model based on a dual-path network, including a data processing module, a model construction module, a model training module, a model evaluation module and a prediction module connected in sequence; the data processing module is used for preprocessing source data and dividing the preprocessed source data into a training set, a validation set and a test set; the model construction module is used for constructing a deep learning model according to a dual-path network; the model training module is used for training the deep learning model according to the training set, adjusting the model parameters of the deep learning model based on the validation set, and obtaining optimized model parameters and an optimized deep learning model; the model evaluation module is used for performing performance evaluation on the optimized deep learning model based on the test set and the optimized model parameters, and obtaining a trained dual-path network model; and the prediction module is used for inputting a tumor data set to be predicted into the trained dual-path network model to predict the radiotherapy response of the tumor.

[0021] The embodiments of the present application have at least the following advantages:

[0022] The embodiment of the present application provides a method and model for predicting tumor radiotherapy response based on a dual-path network. The method includes the following steps: first, preprocessing the source data and dividing the preprocessed source data into a training set, a validation set, and a test set; then, building a deep learning model based on the dual-path network; next, training the deep learning model based on the training set, adjusting the model parameters of the deep learning model based on the validation set, and obtaining optimized model parameters and an optimized deep learning model; then, evaluating the performance of the optimized deep learning model based on the test set and the optimized model parameters to obtain a trained dual-path network model; finally, inputting the tumor data set to be predicted into the trained dual-path network model to predict the tumor's radiotherapy response. The present application proposes a method and model for predicting tumor radiotherapy response based on a dual-path network. First, by constructing a network architecture based on dual-path networks (DPNs), introducing a shallowly optimized DPN structure, and combining it with a customized network output layer (Pixel-wise Output Layer), the adaptability and accuracy of the network in tumor image analysis are enhanced. Secondly, this application integrates multimodal medical imaging data, combines data preprocessing and batch stacking operations, and uses the leave-one-out cross-validation principle to reasonably divide the training set, validation set, and test set, thereby effectively improving the generalization ability and robustness of the model. Finally, by using the mean square error (MSE) and the coefficient of determination R 2 The model's performance was evaluated using a variety of indicators, verifying the effectiveness, accuracy, and tuning capabilities of the proposed model. The model provided in this application can accurately predict tumor responses after radiotherapy, thereby intelligently assisting physicians in adjusting radiotherapy doses and developing personalized, precise radiotherapy plans for patients, reducing unnecessary radiotherapy side effects and improving physicians' work efficiency and the quality of diagnosis and treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] One or more embodiments are exemplarily described by the pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Unless otherwise stated, the pictures in the drawings do not constitute proportional limitations.

[0024] Figure 1 A schematic flow chart of a method for predicting tumor radiotherapy response based on a dual-path network provided in one embodiment of the present application.

[0025] Figure 2 A schematic diagram of the data preprocessing process of a tumor radiotherapy response prediction method based on a dual-path network provided in one embodiment of the present application.

[0026] Figure 3 A network architecture diagram of a dual-path network provided in one embodiment of the present application.

[0027] Figure 4 A schematic diagram of the structure of a tumor radiotherapy response prediction model based on a dual-path network provided in one embodiment of the present application. DETAILED DESCRIPTION

[0028] As can be seen from the background, current tumor radiotherapy settings still rely primarily on the physician's clinical experience. Radiotherapy often produces a certain amount of side effects for patients, and is highly individual and subject to chance. Relying solely on physician experience cannot guarantee high accuracy, high reliability, and low side effects. Therefore, precise setting of tumor radiotherapy doses is crucial for achieving personalized, precision chemoradiotherapy.

[0029] At present, the dose setting in tumor radiotherapy mainly depends on the experience of the physician, so the individual differences and randomness are large, and high precision, low reliability and low side effects cannot be continuously guaranteed. The dual path network (DPN) is a dual-channel neural network in the field of deep learning. It is a parallel network of the residual neural network (ResNet) and the dense connection network (DenseNet). It presents a new connection path topology inside. The equivalence of ResNet and DenseNet is revealed through the high-order recurrent neural network (HORNN) framework, and it points out that ResNet supports feature reuse and DenseNet supports the ability to explore new features. Therefore, DPN is actually a new dual-path network structure. It is a family of convolutional neural networks that includes residual similarity paths and dense connection similarity paths (residual networks are essentially dense connection networks with shared connections). By connecting the two in parallel and combining their respective advantages, performance optimization is achieved. Therefore, DPN, as a dual-path network, can process image features efficiently and accurately. Therefore, this application takes DPN as the main starting point and does a lot of customized work for accurate prediction before and after tumor radiotherapy.

[0030] In response to the shortcomings of the existing technology, an embodiment of the present application provides a method and model for predicting tumor radiotherapy response based on a dual-pathway network. The method includes the following steps: first, preprocessing the source data and dividing the preprocessed source data into a training set, a validation set, and a test set; then, constructing a deep learning model based on the dual-pathway network; next, training the deep learning model based on the training set and adjusting the model parameters of the deep learning model based on the validation set to obtain optimized model parameters and an optimized deep learning model; then, performing a performance evaluation on the optimized deep learning model based on the test set and the optimized model parameters to obtain a trained dual-pathway network model; finally, inputting the tumor data set to be predicted into the trained dual-pathway network model to predict the tumor's radiotherapy response. The purpose of the present application is to provide a method and model for predicting tumor radiotherapy response based on a dual-pathway network. The method and model can accurately predict the tumor's response after radiotherapy at the pixel level, thereby assisting radiotherapy physicians in adjusting radiotherapy doses, reducing the harm of radiotherapy side effects to patients, and improving the physician's work efficiency.

[0031] The following detailed description of the various embodiments of the present application is provided in conjunction with the accompanying drawings. However, those skilled in the art will appreciate that many technical details are provided in the various embodiments of the present application to facilitate a better understanding of the present application. However, even without these technical details and the various variations and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0032] See Figure 1 The present invention provides a method for predicting tumor radiotherapy response based on a dual-path network, the method comprising the following steps:

[0033] Step S101: pre-process the source data and divide the pre-processed source data into a training set, a validation set and a test set.

[0034] Step S102: Build a deep learning model based on the dual-path network.

[0035] Step S103: train the deep learning model based on the training set, and adjust the model parameters of the deep learning model based on the validation set to obtain optimized model parameters and an optimized deep learning model.

[0036] Step S104: Based on the test set and the optimized model parameters, the performance of the optimized deep learning model is evaluated to obtain a trained dual-path network model.

[0037] Step S105: input the tumor data set to be predicted into the trained dual-pathway network model to predict the radiotherapy response of the tumor.

[0038] DPN is a dual-channel neural network in the field of deep learning. It is a parallel network of residual neural network (ResNet) and dense connection network (DenseNet). It presents a new connection path topology inside, reveals the equivalence of ResNet and DenseNet through the high-order recurrent neural network (HORNN) framework, and points out the ability of ResNet to support feature reuse and the ability of DenseNet to support the exploration of new features. Therefore, DPN is actually a new dual-path network structure. It is a family of convolutional neural networks that includes residual similarity paths and dense connection similarity paths (residual networks are essentially dense connection networks with shared connections). By connecting the two in parallel and combining their respective advantages, performance optimization is achieved. Therefore, DPN, as a dual-path network, can process image features efficiently and accurately. Therefore, this application proposes a tumor radiotherapy response prediction method and model based on a dual-path network. This method and model can accurately predict the response of tumors after radiotherapy at the pixel level, thereby assisting radiotherapy physicians in adjusting radiotherapy doses, reducing the harm of radiotherapy side effects to patients, and improving the work efficiency of physicians.

[0039] Specifically, in step S101, the source data (the patient's original tumor dataset) is preprocessed. The pre-radiotherapy SUV value of the tumor is extracted from the patient's PET / CT image source data, denoted as prePET; the mid-radiotherapy SUV value is extracted, denoted as midPET; and the radiation dose data is extracted, labeled Dose. These three sets of raw data are then sliced ​​along the z-axis of the PET / CT scan and converted into a two-dimensional matrix. The prePET and Dose slices are then fused into an RGB image, with prePET serving as the R channel and Dose serving as the G channel. The resulting RGB image is named prePET. Simultaneously, a grayscale image is generated from the midPET slice, named midPET. The processed prePET and midPET slice sets are then stacked according to the batch size to form several batches. Using a leave-one-out cross-validation strategy, one patient is assigned as a test subject in each validation round, ensuring that all patients serve as test subjects in a given round. Next, patients are categorized into three groups based on their response to radiation therapy: high response, moderate response, and low response. An equal proportion of patients and their data from each group are selected as the validation set. Finally, the remaining patients and their data were used to construct the training set.

[0040] The standard uptake value (SUV), mentioned in the preceding steps, is a parameter used in PET imaging to quantify the distribution of radiotracers in the body. It represents the radioactivity concentration within the region of interest (ROI) per unit body weight. SUV is normalized to facilitate standardized comparison of results across individuals and scanning conditions. It is widely used in tumor diagnosis, therapeutic efficacy monitoring, and disease staging. Its calculation takes into account ROI radioactivity concentration, injected dose, and body weight. It is influenced by multiple factors, including patient metabolism and scanning conditions, and serves as an important indicator for assisting clinical decision-making.

[0041] In some embodiments, the number of patients in the training, validation, and test sets, as well as the dataset size, should be balanced. Generally, for smaller datasets, a ratio of 21:3:1 for training: validation: test sets is sufficient for leave-one-out cross-validation. For larger datasets, a ratio of 6:2:2 is more reasonable. This is done to prevent poor model performance due to insufficient training data or poor generalization due to excessive training data.

[0042] After dividing the training set, validation set and test set, a deep learning model is constructed based on the DPN network architecture; then the deep learning model is trained based on the training set, and the model parameters of the deep learning model are adjusted based on the results of the validation set to achieve model optimization.

[0043] The loss function used in the model training process is the mean square error (MSE) loss function. The pixel-level prediction output of the model is obtained through the forward propagation of the deep learning model, and the MSE is calculated. The calculation formula is:

[0044]

[0045] Among them, m represents the number of pixels in the first dimension, n represents the number of pixels in the second dimension, and midPET i，j is the true value of the data, is the model prediction value, and its calculation result is used for back propagation to update the model parameters; the model parameters are adjusted according to the MSE value after model training convergence, so that the MSE loss value after convergence during model verification is as small as possible, and finally the model parameters with the smallest MSE loss value after convergence are selected and imported into the deep learning model, and then the model prediction performance is tested using the test set.

[0046] In some embodiments, step S101 includes preprocessing source data, stacking data in batches, and dividing the data into a training set, a validation set, and a test set, including:

[0047] Step S1011: perform grayscale processing, RGB channel fusion and other operations on the source data.

[0048] Step S1012, data batch stacking is performed on the prepared source data.

[0049] Step S1013, the training set, the validation set and the test set are divided.

[0050] In some embodiments, in step S1011, the source data is subjected to grayscale image processing, RGB channel fusion and the like, including: the source data is subjected to grayscale image processing, RGB channel fusion and the like, that is, the tumor pre-radiotherapy image standardized uptake value (SUV) from the patient PET / CT image source data is recorded as prePET, the mid-radiotherapy image SUV value is recorded as midPET, and the radiotherapy dose is recorded as Dose. Then, the three groups of original data are sliced along the z-axis of PET / CT scanning to be converted into two-dimensional matrices, and then the prePET and Dose slices are subjected to RGB channel fusion, that is, prePET is taken as R channel data, Dose is taken as G channel data to generate an RGB image, which is recorded as prePET. The midPET is taken as a grayscale image color channel to generate a grayscale image, which is recorded as midPET.

[0051] In some embodiments, in step S1012, the data batch stacking includes stacking the prepared prePET and midPET slice sets according to the Batchsize set by the deep learning to form a plurality of batches, so as to prepare for subsequent input model training. The Batchsize is the number of samples contained in each batch.

[0052] In some embodiments, in step S1013, the training set, the validation set and the test set are divided, including: based on the Leave-One-Out-Cross-Validation principle, first, one test patient is allocated for each cross-validation round, so that all patients and their data sets are used as test patients in a cross-validation round, and the traversal is completed; second, according to the response degree of the patient to the radiotherapy, the patients are classified into high response, medium response and low response, and the same number of patients and their data sets are selected from each category of patients as the validation set of the model; finally, the remaining all patients and their data sets in the data set are used as the training set of the model.

[0053] In some embodiments, in step S102, the deep learning model is constructed based on the network architecture of DPN, including:

[0054] Step S1021, the network main architecture of DPN is constructed;

[0055] Step S1022: Based on the DPN network architecture constructed in step S1021, the training set and the test set are converted into structured image data suitable for DPN input, and the image data is input into the DPN network main architecture for training. The model parameters are adjusted to achieve the optimal value, and finally a deep learning model is obtained based on the lowest loss value obtained by inputting the validation set;

[0056] The main architecture of the DPN network includes multiple DualPath Block modules and two classifier layers; the network body of the DPN obtains the output value through forward propagation. After the output is adaptively pooled, it is further processed by the convolution classifier and the fully connected layer to obtain the final predicted feature map, and the feature map is converted to calculate the loss value and related prediction evaluation indicators; before the image data is input into the main architecture of the DPN network for training, it also includes: converting the image data into a tensor, and performing necessary preprocessing and format conversion on the tensor as the input of the main architecture of the DPN network.

[0057] Specifically, this application constructs a pre-architecture for the DPN network. Based on this architecture, the training and test sets are converted into structured image data suitable for DPN input, and then input into the main DPN network architecture for training. During the training process, the model parameters are adjusted to achieve optimal performance, and the deep learning model is ultimately determined based on the lowest loss value obtained on the validation set input.

[0058] In some embodiments, step S105 of inputting the tumor dataset to be predicted into the trained deep learning model includes:

[0059] Step S1051 : pre-process the tumor dataset to be predicted to obtain a processed tumor dataset.

[0060] Step S1052: Input the preprocessed tumor data set into the trained deep learning model to predict the tumor radiotherapy response.

[0061] In some embodiments, in step S1051, the tumor dataset to be predicted is preprocessed, including: grayscale image processing, RGB channel fusion, data batch stacking and other operations on the tumor dataset to be predicted, so that the dimension of the preprocessed tumor dataset to be tested is the same as the dimension of the tumor dataset in the training set, validation set and test set.

[0062] Specifically, the preprocessing process for the tumor dataset to be predicted is consistent with the preprocessing steps for the training, validation, and test sets, including grayscale image processing, RGB channel fusion, and data batch stacking. The dimensions of the preprocessed tumor dataset to be tested should be consistent with the dimensions of the tumor datasets in the training, validation, and test sets. The preprocessed data is then input into the trained deep learning model to generate predictions, thereby determining the tumor's response to radiotherapy.

[0063] See Figure 2 , an embodiment of the present application provides a data preprocessing flow diagram of a tumor radiotherapy response prediction method based on a dual-path network, comprising the following steps: first, extracting relevant features from the patient's PET / CT image source data, including the tumor's pre-radiotherapy image SUV value (denoted as prePET), mid-radiotherapy image SUV value (denoted as midPET), and radiotherapy dose (denoted as Dose), and slicing these data along the z-axis of the PET / CT scan and converting them into a two-dimensional matrix; then, performing specific fusion processing on the slices, wherein prePET is used as R channel data and Dose is used as G channel data to generate an RGB image and denoted as prePET, while midPET is used as a grayscale image color channel to generate a grayscale image and denoted as midPET; then, stacking the processed prePET and midPET slice sets into several batches according to the batchsize required by deep learning (such as batchsize = 4) to meet the input layer requirements of the DPN network.

[0064] Specifically, tumor data is extracted from tumor imaging data, and source data preprocessing, data batch stacking and other operations are performed to finally obtain a data set suitable for input into a deep learning model.

[0065] In some embodiments, dividing the training set, the validation set, and the test set further includes: allocating the training set, the validation set, and the test set according to a leave-one-out cross-validation principle and the size of the data set.

[0066] Specifically, for the division of the data set, first, a test patient is assigned to each round of cross-validation, so that all patients and their data sets are used as test patients in a certain round of cross-validation to complete the traversal; secondly, the patients are classified into high response, medium response, and low response according to the degree of their response to radiotherapy, and the same number of patients and their data sets are selected from each category of patients as the validation set of the model; finally, all the remaining patients and their data sets in the data set are used as the training set of the model.

[0067] See Figure 3, this application example provides a network architecture diagram of a dual path network (DPN). The main network architecture of the DPN includes multiple Dual Path Block modules and two classifier layers; the network body of the DPN obtains the output value through forward propagation, and after the output is adaptively pooled, it is further processed by the convolution classifier and the fully connected layer to obtain the final predicted feature map, and the feature map is converted to calculate the loss value and related prediction evaluation indicators; the network architecture of the DPN includes converting the training set and the test set into structured image data suitable for DPN input, and inputting the image data into the main network architecture of the DPN for training, achieving the optimal by adjusting the model parameters, and finally obtaining a deep learning model based on the lowest loss value obtained by the input verification set.

[0068] After constructing the main network architecture of the dual-path network, and before converting the training set and the test set into structured image data suitable for the dual-path network input based on the main network architecture of the dual-path network, and inputting the structured image data into the main network architecture of the dual-path network for training, it also includes: optimizing the main network architecture of the dual-path network to obtain a shallow dual-path network model.

[0069] Specifically, a shallower, custom DPN model is used within the DPN network architecture to mitigate overfitting. This approach includes: reducing the number of Dual Path Block modules within the DPN network to construct a shallower network structure, while also reducing the number of channels in each module to reduce the dimensionality of the feature maps; using a smaller initial convolution kernel size to reduce the number of parameters; and lowering the width expansion factor (BW factor) to reduce the network width. This method also utilizes grouped convolution to reduce parameters and computational complexity, further reducing model complexity and mitigating overfitting.

[0070] The DPN structure proposed in this application is an innovative dual-path network structure with low model complexity, reduced risk of overfitting, and an innovative network output layer that can perform pixel-level prediction tasks on tumor images.

[0071] In some embodiments, the network architecture of the DPN also includes: using a specially designed network output layer, namely the Pixel-wise Output Layer, that is, a customized output layer, and adding a 1×1 convolution layer at the end of the network to perform feature mapping on each pixel point, thereby outputting a feature map with the same spatial resolution as the input image to ensure the accuracy of pixel-level prediction. Subsequently, the convolution features are converted into a specified number of output channels through a fully connected layer to obtain a specific prediction value for each pixel, and an adaptive pooling layer is introduced to adapt the output size to input images of different sizes. Finally, the output result is converted into the target shape through a reshape operation to meet the task requirements of pixel-level prediction. This process ensures that the output result has the same spatial resolution as the input image to achieve accurate pixel-level prediction.

[0072] See Figure 4 , an embodiment of the present application also provides a tumor radiotherapy response prediction model based on a dual-path network, comprising: a data processing module 101, a model construction module 102, a model training module 103, a model evaluation module 104, a prediction module 105 and a data processing unit in the prediction module 105, which are connected in sequence; wherein the data processing module 101 is used to preprocess the source data and divide it into a training set, a validation set and a test set; the model construction module 102 is used to construct a deep learning model based on the DPN network architecture; the model training module 103 is used to train the deep learning model based on the training set, and adjust the model parameters of the deep learning model based on the validation set to obtain optimized model parameters and an optimized deep learning model; the model evaluation module 104 is used to perform performance evaluation on the optimized deep learning model based on the test set and the optimized model parameters to obtain a trained DPN model; the prediction module 105 and its internal sub-module data processing unit are used to preprocess the tumor data set to be predicted and input it into the above-mentioned trained DPN model to predict the radiotherapy response of the tumor.

[0073] Based on the above technical solution, an embodiment of the present application provides a tumor radiotherapy response prediction method and model based on a dual-pathway network, the method comprising the following steps: first, preprocessing the source data, and dividing the preprocessed source data into a training set, a validation set, and a test set; then, constructing a deep learning model based on the dual-pathway network; next, training the deep learning model based on the training set, and adjusting the model parameters of the deep learning model based on the validation set to obtain optimized model parameters and an optimized deep learning model; then, based on the test set and the optimized model parameters, the performance of the optimized deep learning model is evaluated to obtain a trained dual-pathway network model; finally, the tumor data set to be predicted is input into the trained dual-pathway network model to predict the radiotherapy response of the tumor.

[0074] The embodiment of the present application proposes a method for predicting tumor radiotherapy response based on a dual-path network. First, by constructing a network architecture based on DPN, a shallowly optimized DPN structure is introduced, combined with a customized Pixel-wise OutputLayer, to enhance the adaptability and accuracy of the network in tumor image analysis. Secondly, the present application integrates multimodal medical imaging data, combines data preprocessing and batch stacking operations, and uses the leave-one-out cross-validation principle to reasonably divide the training set, validation set, and test set, thereby effectively improving the generalization ability and robustness of the model. Finally, the performance of the model is evaluated by using the MSE and determination coefficient R2 indicators to verify the effectiveness, accuracy, and tuning ability of the proposed model. The model provided in this application can accurately predict the response of tumors after radiotherapy, thereby intelligently assisting physicians in adjusting radiotherapy doses, formulating personalized and precise radiotherapy plans for patients, reducing unnecessary radiotherapy side effects, and improving the work efficiency and quality of diagnosis and treatment of physicians.

[0075] Those skilled in the art will appreciate that the above-described embodiments are specific examples for implementing the present application, and that in actual applications, various changes in form and detail may be made thereto without departing from the spirit and scope of the present application. Any person skilled in the art may make changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be subject to the scope defined in the claims.

Claims

1. A method for predicting tumor radiotherapy response based on a dual-path network, characterized in that: The following steps are involved: Preprocess the source data and divide the preprocessed source data into training set, validation set and test set; Build a deep learning model based on a dual-path network; Training the deep learning model based on the training set, and adjusting model parameters of the deep learning model based on the validation set to obtain optimized model parameters and an optimized deep learning model; Based on the test set and the optimized model parameters, a performance evaluation is performed on the optimized deep learning model to obtain a trained dual-path network model; Inputting the tumor data set to be predicted into the trained dual-pathway network model to predict the tumor's radiotherapy response; Based on the dual-path network, a deep learning model is constructed, which includes the following steps: Construct the main network architecture of the dual-path network; the main network architecture of the dual-path network includes multiple Dual PathBlock modules and two classifier layers. The main network of the dual-path network obtains the output value through forward propagation. After the output is adaptively pooled, it is further processed using a convolutional classifier and a fully connected layer to obtain the final predicted feature map. The feature map is then transformed and the loss value and related prediction evaluation indicators are calculated. Based on the main network architecture of the dual-path network, the training set and test set are converted into structured image data suitable for the dual-path network input. The structured image data is then input into the main network architecture of the dual-path network for training. The model parameters are adjusted to achieve the optimal value. Finally, the deep learning model is obtained based on the lowest loss value obtained by inputting the validation set. Before inputting structured image data into the main network architecture of the dual-path network for training, it also includes: Convert structured image data into tensor, preprocess and convert the tensor format as the input of the main network architecture of the dual-path network; The main network architecture of the dual-path network also includes a specially designed network output layer; The specially designed network output layer is achieved by customizing the output layer; the customization includes: adding a 1×1 convolution layer at the end of the network to perform feature mapping on each pixel point, thereby outputting a feature map with the same spatial resolution as the input image to ensure the accuracy of pixel-level prediction; then, converting the convolution features into a specified number of output channels through a fully connected layer to obtain a specific prediction value for each pixel, and introducing an adaptive pooling layer to adapt the output size to input images of different sizes; finally, converting the output result into a target shape through a reshape operation to meet the task requirements of pixel-level prediction. This process ensures that the output result has a spatial resolution consistent with the input image, so as to achieve accurate pixel-level prediction.

2. The method for predicting tumor radiotherapy response based on a dual-path network according to claim 1, characterized in that: Preprocessing the source data includes the following steps: The source data is processed in grayscale, fused into RGB channels, and stacked in batches. The grayscale image processing includes: obtaining the standard uptake value (SUV) of the tumor image before radiotherapy from the patient's PET / CT image source data, that is, the SUV value of the early radiotherapy image is recorded as prePET, the SUV value of the mid-radiotherapy image is recorded as midPET, and the radiotherapy dose is recorded as Dose; then, these three sets of original data are sliced ​​along the z-axis of the PET / CT scan and converted into a two-dimensional matrix; The RGB channel fusion includes: performing RGB channel fusion on the prePET and Dose slices, that is, using prePET as R channel data and Dose as G channel data to generate an RGB image, which is recorded as prePET; and using midPET as a grayscale image color channel to generate a grayscale image, which is recorded as midPET; The data batch stacking includes: stacking the prepared prePET and midPET slice sets into several batches according to the batch size set for deep learning, in preparation for subsequent input model training, where the batch size is the number of samples included in each batch.

3. The method for predicting tumor radiotherapy response based on a dual-path network according to claim 1, characterized in that: Divide the preprocessed source data into training set, validation set, and test set, including: According to the principle of leave-one-out cross-validation, first, the test set of each round of cross-validation is assigned to a test patient to ensure that all patients and their datasets can be used as test sets in a certain round of cross-validation to achieve full coverage; second, according to the degree of their response to radiotherapy, patients are divided into three categories: high response, moderate response, and low response, and the same number of patients and their datasets are selected from each category to form the validation set of the model; finally, the remaining unselected patients and their datasets are used as the training set of the model.

4. The method for predicting tumor radiotherapy response based on a dual-path network according to claim 1, characterized in that: After constructing the main network architecture of the dual-path network, and before converting the training set and the test set into structured image data suitable for the dual-path network input based on the main network architecture of the dual-path network, and inputting the structured image data into the main network architecture of the dual-path network for training, the method further includes: optimizing the main network architecture of the dual-path network to obtain a shallow dual-path network model; wherein, The optimization process includes: reducing the number of Dual Path Block modules in the dual-path network, building a shallow network structure, and reducing the number of channels in each module to reduce the dimension of the feature map; using a smaller initial convolution kernel to reduce the number of parameters; and reducing the width expansion factor bw_factor to reduce the width of the network.

5. The method for predicting tumor radiotherapy response based on a dual-path network according to claim 1, characterized in that: The deep learning model is trained based on the training set, and model parameters of the deep learning model are adjusted based on the validation set to obtain optimized model parameters and an optimized deep learning model, including: The pixel-level prediction output of the model is obtained through forward propagation, and the loss function is calculated. The loss function is expressed in mean square error, and its calculation formula is: Among them, MSE is the mean square error, m represents the number of pixels in the first dimension, n represents the number of pixels in the second dimension, and midPET i,j is the true value, is the model prediction value, and its calculation result is used for back propagation to update the model parameters; After each training, the model performance is evaluated based on the validation set, and the goodness of fit of the model is measured by calculating the coefficient of determination. The coefficient of determination is expressed as R², and its formula is: Among them, m represents the number of pixels in the first dimension, n represents the number of pixels in the second dimension, and midPET i,j is the true value of the data, To predict the value of the model, the hyperparameters of the model are adjusted by evaluation on the validation set. The hyperparameters include the learning rate and the regularization term, and finally the optimized model parameters and deep learning model are obtained, thereby improving the prediction accuracy of tumor radiotherapy response.

6. The method for predicting tumor radiotherapy response based on a dual-path network according to claim 1, characterized in that: Based on the test set and the optimized model parameters, the performance of the optimized deep learning model is evaluated to obtain a trained dual-path network model, including: The same evaluation metrics as those used for training and validation are used, including mean square error and coefficient of determination. A test is performed after each epoch is completed and the model parameters are updated. This test does not affect the gradient of the model.

7. The method for predicting tumor radiotherapy response based on a dual-path network according to claim 1, characterized in that: Inputting the tumor dataset to be predicted into the trained dual-pathway network model to predict the tumor's radiotherapy response includes: Preprocessing the tumor dataset to be predicted, wherein the preprocessing includes grayscale image processing, RGB channel fusion, and data batch stacking operations, so that the dimension of the preprocessed tumor dataset to be tested is the same as the dimension of the tumor datasets in the training set, validation set, and test set, thereby obtaining a preprocessed tumor dataset; The preprocessed tumor data set is input into the trained dual-pathway network model to predict the tumor's radiotherapy response.

8. A tumor radiotherapy response prediction model based on a dual-pathway network, the model being used to implement the tumor radiotherapy response prediction method based on a dual-pathway network as claimed in any one of claims 1 to 7, characterized in that: It includes a data processing module, a model building module, a model training module, a model evaluation module and a prediction module connected in sequence; The data processing module is used to preprocess the source data and divide the preprocessed source data into a training set, a verification set and a test set; The model building module is used to build a deep learning model based on the dual-path network; The model training module is used to train the deep learning model according to the training set, and adjust the model parameters of the deep learning model based on the validation set to obtain optimized model parameters and an optimized deep learning model; The model evaluation module is used to perform performance evaluation on the optimized deep learning model based on the test set and the optimized model parameters to obtain a trained dual-path network model; The prediction module is used to input the tumor data set to be predicted into the trained dual-pathway network model to predict the radiotherapy response of the tumor.

Images