Radioactive xerostomia depth prediction system based on multi-source channel fusion

Abstract

The invention discloses a radioactive xerostomia depth prediction system based on multi-source channel fusion, and relates to the technical field of image processing, and the system comprises a preprocessing module which is used for obtaining a CT image, a Dose distribution image, GTV and parotid gland structure data and corresponding follow-up visit information before radiotherapy of a head and neck cancer case, carrying out data preprocessing, and dividing a data set; the multi-source data fusion module is divided into a 3D Multi-GTVCNN model and a 3D Multi-PACNN model which are respectively used for fusing CT, Dose and GTV data of any size and CT, Dose and parotid gland data of the same size; the decision fusion module is used for fusing prediction results of the two multi-source data modules; and the prediction module is used for predicting the xerostomia. According to the method, the GTV contour is innovatively introduced as input, the models are trained according to different data characteristics, and the prediction performance is improved through decision fusion, so that the models can extract richer characteristics, the error-tolerant rate is increased, and the accuracy and stability of radioactive xerostomia prediction are improved.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to a radiation xerostomia depth prediction system based on multi-source channel fusion. Background Art

[0002] Radiotherapy is one of the treatments for head and neck cancer. However, a considerable number of patients will experience varying degrees of dry mouth symptoms after receiving radiotherapy, and this dry mouth symptom cannot be relieved by drinking water. Dry mouth will further affect taste, swallowing, and speech functions. This is because the patient's salivary glands are damaged after radiation exposure, causing changes in the nature, quantity, and composition of saliva secretion. This side effect will begin to appear during radiotherapy, and it will not be partially recovered until half a year to a year after the end of radiotherapy. Some patients will continue to have dry mouth for many years or even for life, seriously affecting the patient's quality of life. Therefore, predicting dry mouth before radiotherapy can help doctors adjust the radiotherapy plan in time and improve the patient's quality of life.

[0003] Before radiotherapy, doctors will perform CT (computed tomography) positioning on patients to determine the location and size of the tumor and outline the target area and organs at risk for treatment. The parotid gland is one of the important organs at risk for head and neck cancer patients during radiotherapy. It is the most important salivary gland in the human body. Then the physicist will formulate a radiotherapy plan based on the above information, including radiation dose and irradiation angle. According to the radiotherapy plan, the dose distribution can be calculated and a Dose (dose image) can be generated. The Dose image shows the radiation dose received by different areas. These imaging data not only provide detailed anatomical structure and tumor location, but also provide key support for the implementation of precision radiotherapy and efficacy evaluation.

[0004] In order to accurately predict the occurrence of radiation xerostomia, Bohua Wan et al. trained a 3D residual CNN model based on CT data; Xushi Zhang et al. trained an SVM model based on deep learning dosimetry and deep learning radiomics; Kuo Men et al. trained a 3D CNN model based on CT, Dose, parotid gland contour, and submandibular gland contour; Annarita Fanizzi et al. trained a multimodal model based on clinical characteristics, dose characteristics, and radiomics characteristics. These previous studies considered the complementary information between multi-source data and achieved higher prediction performance compared to single input data. However, they ignored that the location and size of the tumor are also one of the factors that cause patients to have radiation xerostomia. The location and size information of the tumor is generally represented by the GTV (Gross Tumor Volume, tumor target area) contour structure. Therefore, how to use CT images, Dose images, GTV contour structure images, and parotid gland contour structure images at the same time to avoid excessive resource consumption and long training time caused by inputting the entire head and neck slices while improving the prediction accuracy of radiation xerostomia is a technical problem that needs to be solved in this field. Summary of the invention

[0005] In order to solve the above technical problems, this application proposes the following technical solutions:

[0006] In a first aspect, the present application provides a radiation xerostomia depth prediction system based on multi-channel fusion, comprising:

[0007] The preprocessing module is used to obtain CT image data, Dose image data, GTV and parotid gland structure image data drawn by doctors, and corresponding follow-up information of head and neck cancer cases before radiotherapy, and perform different data preprocessing according to the characteristics of GTV and parotid gland. The development set and test set are divided according to the time when the patient receives treatment, and random stratified sampling is performed at a ratio of 4:1 to divide the development set into a training set and a validation set.

[0008] The multi-source data fusion module is used to extract and fuse the features of CT, Dose, GTV data and CT, Dose, parotid gland data on the 3D Multi-GTVCNN model and the 3D Multi-PACNN model, respectively, to give full play to the advantages of each data. The training set and the validation set are used to train the 3D Multi-GTVCNN model and the 3D Multi-PACNN model and adjust the parameters respectively;

[0009] The decision fusion module is used to fuse the original output scores of the 3D Multi-GTVCNN model and the 3D Multi-PACNN model using the naive Bayes algorithm. The original output score represents the confidence of the model for each category. The higher the original output score, the more the model tends to believe that the input data belongs to this category.

[0010] The prediction module is used to input the test set data into 3DMulti-GTVCNN and 3DMulti-PACNN to determine the model parameters, obtain the original output scores of the two models respectively, and then use the trained naive Bayes model to fuse the results of the two models to complete the prediction of radiation xerostomia;

[0011] In one possible implementation, the preprocessing module includes:

[0012] The resampling unit is used to normalize the voxels of different sizes in the CT image and the corresponding dose distribution image to 1mm*1mm*3mm;

[0013] The normalization unit is used to convert the characteristic values ​​of the samples to the same dimension, that is, to map the characteristic values ​​to the interval [0, 1] using the maximum and minimum value normalization method;

[0014] The GTV contour clipping unit is used to intercept all slices in the area where the GTV exists. CT and Dose respectively intercept the slices corresponding to the GTV and clip the minimum ROI at the same time. The minimum ROI here refers to the minimum matrix area that retains key pixels and excludes redundant zero pixels, which greatly reduces the data size and reduces the consumption of computing resources.

[0015] The parotid gland contour clipping unit is used to capture all slices of the parotid gland area. CT and Dose respectively capture the slices corresponding to the parotid gland and clip the minimum ROI at the same time.

[0016] Image size unification unit, used to unify the parotid contour image and its corresponding CT and Dose images to a maximum size of 248*248, and to unify the depth to a maximum depth of 29 layers. For the size unification of height and width, zero pixels are evenly filled on the four sides of the image. For the size unification of depth, all-zero slices are added after the last slice until the final size reaches 248*248*29;

[0017] The data enhancement unit is used to enhance the processed image data by rotating it in any direction at any angle within 10°, randomly reducing or enlarging it between 0.5 and 1.5 times, and arbitrarily translating it in the horizontal or vertical direction within 10 pixel units. The changes in data enhancement for each set of data on multiple modal images are the same. Data enhancement is only for the training set, and the validation set and test set are not processed.

[0018] In a possible implementation, the multi-source data fusion module includes:

[0019] 3D Multi-GTVCNN model, input CT of any size, Dose and GTV contours are respectively passed through a convolution layer with a convolution kernel size of 3*5*5 and a stride of 1. LN (Layer Normalization) is used instead of BN (Batch Normalization) in traditional CNN to ensure that the input image can be of any size. After fusing the multiple modal feature maps extracted by the first convolution layer, they are input into a convolution layer with a convolution kernel size of 1*1*1 and a stride of 1. Subsequently, the convolution layer with a convolution kernel size of 1*3*3 and a stride of 1 and a convolution layer with a convolution kernel size of 1*3*3 and a stride of 2 are input. Finally, the average pooling layer converts the feature map into a feature vector, and after passing through the fully connected layer, the original output scores of the two types of labels are output.

[0020] 3D Multi-PACNN model, input CT with a size of 248*248*29, Dose and parotid gland contour pass through a convolution layer with a convolution kernel size of 3*5*5 and a stride of 1 respectively. After fusing the multiple modal feature maps extracted by the first convolution layer, they are input into a convolution layer with a convolution kernel size of 1*1*1 and a stride of 1. Subsequently, they are input into a convolution layer with a convolution kernel size of 3*3*3 and a stride of 1 and a convolution layer with a convolution kernel size of 3*3*3 and a stride of 2. Finally, the average pooling layer converts the feature map into a feature vector, and after passing through the fully connected layer, the original output scores of the two types of labels are output;

[0021] The decision fusion unit uses the stacking ensemble learning method to fuse the prediction results of the 3D Multi-GTVCNN model and the 3D Multi-PACNN model. First, the original output scores of the two models on the training set and the test set are obtained respectively. The original output scores of the two models on the training set are trained by the Naive Bayes algorithm. The Naive Bayes model will learn how to combine the prediction results of the 3D Multi-GTVCNN model and the 3D Multi-PACNN model. The final prediction result is obtained by using the trained Naive Bayes model to fuse the original output scores of the two models on the test set.

[0022] In one possible implementation, the training of the 3D Multi-GTVCNN model uses the Adam optimizer for parameter learning, the initial learning rate is 0.000005, the cosine annealing strategy is used to adjust the learning rate, the batch size is set to 1, the epoch is set to 150, the early stopping strategy is used to avoid model overfitting, and the patience is set to 10. The training of the 3D Multi-PACNN model uses the Adam optimizer for parameter learning, the initial learning rate is 0.0001, the cosine annealing strategy is used to adjust the learning rate, the batch size is set to 4, the epoch is set to 150, the early stopping strategy is used to avoid model overfitting, and the patience is set to 10. The loss function uses the cross-entropy loss function (CrossEntropy Loss), which evaluates the difference between the probability distribution of the model output and the true label by comparing the difference between the true probability distribution and the predicted probability distribution. For the binary classification task, the specific expression is:

[0023]

[0024] in, is the probability that the model predicts that the sample is a positive example, y is the sample label, if the sample is a positive example, the value is 1, otherwise the value is 0. In the embodiment of the present application, the positive example is radiation xerostomia after radiotherapy.

[0025] Calculate the true positive rate TP, false positive rate FP, true negative rate TN, and false negative rate FN corresponding to the model prediction respectively;

[0026] Accuracy calculation unit, used to further calculate the accuracy of the model F1 value in Sensitivity Specificity The ROC curve is generated based on the probability value of the classification result, and the corresponding AUC value is calculated to evaluate the performance of different models.

[0027] In the embodiment of the present application, 3DCNN is used to perform fusion training on multi-source data. Compared with previous studies, the GTV contour is innovatively introduced as input. At the same time, according to different data characteristics, it is proposed to train different models separately to give full play to the advantages of each data. The training method of further improving the prediction performance through decision fusion enables the model to extract richer and more complex features, tolerate the influence of prediction errors of a single model, and improve the accuracy and stability of the prediction of radiation xerostomia. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1A schematic diagram of a radiation xerostomia depth prediction system based on multi-channel fusion provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the neural network structure of the 3D Multi-GTVCNN model provided in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of the neural network structure of the 3D Multi-PACNN model provided in an embodiment of the present application;

[0031] Figure 4 A schematic diagram of a depth prediction model for radiation-induced xerostomia based on multi-channel fusion provided in an embodiment of the present application;

[0032] Figure 5 ROC curves of the 3D Multi-GTVCNN model, the 3D Multi-PACNN model and the fused multi-channel fusion-based radiation xerostomia deep prediction model on the test set provided in the embodiments of the present application. DETAILED DESCRIPTION

[0033] The present solution is described below in conjunction with the accompanying drawings and specific implementation methods.

[0034] Figure 1 A schematic diagram of a radiation xerostomia prediction system based on a multimodal deep learning network provided in an embodiment of the present application, see Figure 1 The radiation xerostomia depth prediction system 100 based on multi-channel fusion in this embodiment includes:

[0035] Preprocessing module 101 is used to obtain CT image data, dose image data, GTV and parotid gland structure image data drawn by doctors and corresponding follow-up information of patients with head and neck cancer before radiotherapy, and perform data preprocessing to obtain training set, validation set and test set

[0036] The multi-source data fusion module 102 is used to extract and fuse the CT, Dose, GTV data and the CT, Dose, parotid gland data on the model 3D Multi-GTVCNN and the model 3D Multi-PACNN, respectively, and train and adjust and improve the parameters of the model 3D Multi-GTVCNN and the model 3D Multi-PACNN using the training set and the validation set;

[0037] A decision fusion module 103 is used to fuse the original output scores of the 3D Multi-GTVCNN model and the 3D Multi-PACNN model using a naive Bayes algorithm, wherein the original output score represents the confidence of the model for each category, and the higher the original output score, the more the model tends to believe that the input data belongs to the category;

[0038] The prediction module 104 is used to input the data of the test set into the 3D Multi-GTVCNN and 3D Multi-PACNN for determining the model parameters, obtain the original output scores of the two models respectively, and then use the trained naive Bayes model to fuse the results of the two models to complete the prediction of radiation xerostomia.

[0039] Obtain CT imaging data, dose imaging data, GTV and parotid gland structure imaging data drawn by doctors, and corresponding follow-up information of head and neck cancer cases before radiotherapy. Randomly stratify and sample the data of 137 head and neck cancer cases collected from September 2020 to June 2023 at the University Town Hospital of Guangdong Provincial Hospital of Traditional Chinese Medicine at a ratio of 4:1 and divide them into training set and validation set. The ratio of positive and negative classes in the training set and validation set is 3:2. Among them, the positive class refers to cases with dry mouth 6 months after the end of radiotherapy, and the negative class refers to cases without dry mouth 6 months after the end of radiotherapy. 43 head and neck cancer cases collected from August 2023 to March 2024 were used as the test set, and the ratio of positive and negative classes in the test set was 4:3. The present invention collects CT images that can be used for tumor localization of patients diagnosed with head and neck cancer before radiotherapy, as well as Dose images developed on this basis and GTV and parotid gland contour images drawn by doctors. The label information is obtained through telephone follow-up six months after the end of radiotherapy, and data with incomplete, unclear images and no follow-up information are excluded.

[0040] Preprocess the raw data. Different raw data have different voxel sizes. Resampling and normalization are conducive to batch operation of the network model. The volume of the parotid gland and tumor accounts for a small proportion. If the slices of the entire head and neck are input, it will lead to huge consumption of computing resources, slow operation speed, and may even fail to run due to insufficient video memory. Only intercepting slices of the GTV or parotid region can greatly reduce the depth size of the image and reduce the amount of calculation. In addition, the location and size of the tumor may vary greatly in different patients, so the number of slices intercepted will be very different. The uniform size may lead to the addition of too much useless information. The location and size of the parotid gland in adults generally do not differ significantly, and the number of slices intercepted is also relatively consistent. The uniform size can improve the efficiency of batch processing and speed up training. Different models process input data differently. Therefore, in order to reduce the differences between data and maximize the extraction of effective feature information, the data must be preprocessed before entering the network.

[0041] The preprocessing module 101 described in this embodiment includes: a resampling unit, which is used to normalize the voxels of different sizes in the CT image and the corresponding dose distribution image to 1mm*1mm*3mm. A normalization unit, which is used to convert the eigenvalues ​​of the sample to the same dimension, that is, to use the maximum and minimum normalization method to map the eigenvalues ​​to the interval [0, 1]. A GTV structure cropping unit, which is used to cut out all slices containing the GTV, and CT and Dose respectively cut out the slices corresponding to the GTV, and at the same time cut out the minimum ROI, and the size of the input model is the original ROI size. A parotid gland structure cropping unit, which is used to cut out all slices containing the parotid gland, and CT and Dose respectively cut out the slices corresponding to the parotid gland, and at the same time cut out the minimum ROI. Image size unification unit, used to unify the parotid contour image and its corresponding CT and Dose images to a maximum size of 248*248, and the depth to a maximum depth of 29 layers. For the size unification of height and width, zero pixels are evenly filled on the four sides of the image. For the size unification of depth, all zero slices are added after the last slice until the final size reaches 248*248*29. Data enhancement unit, used to enhance the processed image data by rotating it in any direction within 10°, randomly reducing or enlarging it between 0.5 and 1.5 times, and arbitrarily translating pixels within 10 pixel units in the horizontal or vertical direction. The changes in data enhancement for each set of data on multiple modal images are the same. Data enhancement is only for the training set, and no processing is performed on the validation set and test set.

[0042] The 3D Multi-GTVCNN and 3D Multi-PACNN models were trained using the training set, and the two models were verified using the validation set. The results were compared and the best performing network model was retained.

[0043] The multi-source data fusion module 102 includes: a 3D Multi-GTVCNN model that accepts input data of any size. The input CT, Dose and GTV contours are respectively passed through a convolution layer (Conv in the figure) with a convolution kernel size of 3*5*5 and a step size of 1. LN (LayerNormalization) is used to ensure that the input image can be of any size. After the multiple modal feature maps extracted by the first convolution layer are fused, they are input into a convolution layer with a convolution kernel size of 1*1*1 and a step size of 1. Subsequently, the convolution layer with a convolution kernel size of 1*3*3 and a step size of 1 and the convolution layer with a convolution kernel size of 1*3*3 and a step size of 2 are input. Finally, the average pooling layer converts the feature map into a feature vector. After passing through the fully connected layer, the original output scores of the two types of labels are output. The 3D Multi-GTVCNN model is as follows: Figure 2 shown.

[0044] Figure 3 The figure shows the 3D Multi-PACNN model provided in an embodiment of the present application. The input CT, Dose and parotid gland contour are respectively passed through a convolution layer with a convolution kernel size of 3*5*5 and a step size of 1. LN is used to ensure that the input image can be of any size. The multi-source data feature map extracted by the first convolution layer is fused and input to a convolution layer with a convolution kernel size of 1*1*1 and a step size of 1. Subsequently, a convolution layer with a convolution kernel size of 3*3*3 and a step size of 1 and a convolution layer with a convolution kernel size of 3*3*3 and a step size of 2 are input. Finally, the average pooling layer converts the feature map into a feature vector, and the original output scores of the two types of labels are output after passing through the fully connected layer.

[0045] The decision fusion module in this embodiment includes: using a stacking algorithm to fuse the prediction results of the 3D Multi-GTVCNN model and the 3D Multi-PACNN model, first obtaining the original output scores of the two models on the training set and the test set respectively, training the original output scores of the two models on the training set by the naive Bayes algorithm, and the naive Bayes model will learn how to combine the prediction results of the 3D Multi-GTVCNN model and the 3D Multi-PACNN model, and obtain the final prediction result by fusing the original output scores of the two models on the test set using the trained naive Bayes model. The decision fusion rather than feature fusion is used to enable each model of the GTV and parotid contour data to be trained and predicted independently without sharing feature space. Different models are used to give full play to their respective advantages. Moreover, the decision fusion can tolerate the influence of wrong decisions of a single model, and the robustness and stability of the prediction are enhanced by integrating the results of 3D Multi-GTVCNN and 3D Multi-PACNN.

[0046] In this embodiment, the training of the 3D Multi-GTVCNN model uses the Adam optimizer for parameter learning, the initial learning rate is 0.000005, the cosine annealing strategy is used to adjust the learning rate, the batch size is set to 1, the epoch is set to 150, the early stopping strategy is used to avoid model overfitting, and the patience is set to 10. The 3D Multi-PACNN model training uses the Adam optimizer for parameter learning, the initial learning rate is 0.0001, the cosine annealing strategy is used to adjust the learning rate, the batch size is set to 4, the epoch is set to 150, the early stopping strategy is used to avoid model overfitting, and the patience is set to 10. The loss function uses the cross entropy loss function (CrossEntropy Loss), which evaluates the difference between the probability distribution output by the model and the true label by comparing the difference between the true probability distribution and the predicted probability distribution. For the binary classification task, the specific expression is:

[0047]

[0048] Among them, is the probability that the model predicts that the sample is a positive example, y is the sample label, if the sample is a positive example, the value is 1, otherwise the value is 0. In this embodiment of the present application, the positive example is radiation xerostomia after radiotherapy.

[0049] Furthermore, the true positive rate TP, false positive rate FP, true negative rate TN, and false negative rate FN corresponding to the model prediction are calculated respectively. The accuracy calculation unit is used to calculate the accuracy of the model F1 value in Sensitivity Specificity The ROC (Receiver Operating Characteristic) curve is generated based on the probability value of the classification result, and the corresponding AUC value is calculated to evaluate the performance of different models.

[0050] In this embodiment, the model with the highest accuracy and AUC of the model 3D Multi-GTVCNN and the model 3D Multi-PACNN on the validation set is taken as the optimal model, and its model parameters are saved. The final prediction result is obtained by fusing the prediction results of the model 3D Multi-GTVCNN and the model 3D Multi-PACNN.

[0051] In order to comprehensively evaluate the performance of the present application, the results of the radiation xerostomia prediction model proposed in previous studies were compared with the results of the present application. The comparison results are shown in Table 1.

[0052] Table 1 Evaluation of the prediction performance of different models for radiation-induced xerostomia

[0053]

[0054] As can be seen from Table 1, the radiation xerostomia depth prediction model based on multi-source channel fusion proposed in the embodiment of the present invention (composed of 3D Muliti-GTVCNN, 3D Multi-PACNN and Naive Bayes decision fusion model) is higher than the models proposed in previous studies in terms of accuracy, AUC, F1 and sensitivity. Although the specificity is slightly lower than the model proposed by Annarita Fanizzi, the overall performance is the best.

[0055] In addition, by Figure 5 The ROC curves drawn show that the 3D Muliti-GTVCNN and 3D Multi-PACNN models proposed in the embodiments of the present invention, which are trained for the different characteristics of GTV and parotid gland, can effectively extract and fuse multi-source data features and achieve good AUC. After decision fusion through the naive Bayes model, the AUC value is significantly improved, indicating that the accuracy of the model after integrating multiple prediction results has been improved. This method optimizes model performance and increases the reliability and stability of prediction.

[0056] In summary, compared with previous studies, the embodiments of the present invention consider the possible influence of the location and size of the tumor on radiation xerostomia, innovatively introduce the GTV contour as input, and provide more effective feature information for the prediction of radiation xerostomia. At the same time, in view of the large difference in the number of GTV slices in different patients and the similar number of parotid slices, it is proposed to use the 3D Muliti-GTVCNN model and the 3D Multi-PACNN model to process data with different characteristics, give play to the advantages of each data, and then further improve the model performance through decision fusion, which not only maximizes the use of image information to reduce the consumption of computing resources but also improves the prediction performance of the model. The proposed deep learning method can extract the features of multi-source data of CT images, Dose images, GTV contour images, and parotid contour images, and can effectively fuse the feature information of multi-source data, and has good performance in the prediction of radiation xerostomia. Compared with the models proposed in previous related studies, the prediction performance of the constructed radiation xerostomia deep model based on multi-source channel fusion is significantly improved.

[0057] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0058] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A radiation xerostomia depth prediction system based on multi-source channel fusion, characterized in that: include: The preprocessing module is used to obtain CT image data, Dose image data, GTV and parotid gland structure image data drawn by doctors, and corresponding follow-up information of head and neck cancer cases before radiotherapy, and perform different data preprocessing according to the characteristics of GTV and parotid gland. The development set and test set are divided according to the time when the patient receives treatment, and random stratified sampling is performed at a ratio of 4:1 to divide the development set into a training set and a validation set. The multi-source data fusion module is used to extract and fuse the features of CT, Dose, GTV data and CT, Dose, parotid gland data on the 3D Multi-GTVCNN model and the 3D Multi-PACNN model, respectively, to give full play to the advantages of each data. The training set and the validation set are used to train the 3D Multi-GTVCNN model and the 3D Multi-PACNN model and adjust the parameters respectively; The decision fusion module is used to fuse the original output scores of the 3D Multi-GTVCNN model and the 3D Multi-PACNN model using the naive Bayes algorithm. The original output score represents the confidence of the model for each category. The higher the original output score, the more the model tends to believe that the input data belongs to this category. The prediction module is used to input the data of the test set into 3D Multi-GTVCNN and 3D Multi-PACNN to determine the model parameters, obtain the original output scores of the two models respectively, and then use the trained naive Bayes model to fuse the results of the two models to complete the prediction of radiation xerostomia.

2. The radiation xerostomia prediction system based on multi-source channel fusion according to claim 1, characterized in that: The preprocessing module comprises: The resampling unit is used to normalize the voxels of different sizes in the CT image and the corresponding Dose image to 1mm*1mm*3mm; The normalization unit is used to convert the characteristic values ​​of the samples to the same dimension, that is, to use the maximum and minimum value normalization method to map the characteristic values ​​to the interval [0, 1]; The GTV contour clipping unit is used to intercept all slices in the area where the GTV exists. CT and Dose respectively intercept the slices corresponding to the GTV and clip the minimum ROI (Region of Interest). The minimum ROI here refers to the minimum matrix area that retains key pixels and excludes redundant zero pixels, which greatly reduces the data size and reduces the consumption of computing resources. The parotid gland contour clipping unit is used to capture all slices of the parotid gland area. CT and Dose respectively capture the slices corresponding to the parotid gland and clip out the minimum ROI at the same time. Image size unification unit, used to unify the parotid gland contour image and its corresponding CT and Dose images to a maximum size of 248*248, and to unify the depth to a maximum depth of 29 layers. For the unification of the height and width, zero pixels are evenly filled on the four sides of the image. For the unification of the depth, all zero slices are added after the last slice until the final size reaches 248*248*29. The data enhancement unit is used to enhance the processed image data by rotating it in any direction at any angle within 10°, randomly reducing or enlarging it between 0.5 and 1.5 times, and arbitrarily translating it in the horizontal or vertical direction within 10 pixel units. The changes in data enhancement for each set of data on multiple modal images are the same. Data enhancement is only for the training set, and the validation set and test set are not processed.

3. The radiation xerostomia prediction system based on multi-source channel fusion according to claim 1, characterized in that: The multi-source data fusion module includes: 3D Multi-GTVCNN model, input CT of any size, Dose and GTV contours are respectively passed through a convolution layer with a convolution kernel size of 3*5*5 and a stride of 1. LN (Layer Normalization) is used instead of BN (Batch Normalization) in traditional CNN to ensure that the input image can be of any size. After fusing the multiple modal feature maps extracted by the first convolution layer, they are input into a convolution layer with a convolution kernel size of 1*1*1 and a stride of 1. Subsequently, the convolution layer with a convolution kernel size of 1*3*3 and a stride of 1 and a convolution layer with a convolution kernel size of 1*3*3 and a stride of 2 are input. Finally, the average pooling layer converts the feature map into a feature vector, and after passing through the fully connected layer, the original output scores of the two types of labels are output. 3D Multi-PACNN model, input CT with size of 248*248*29, Dose and parotid gland contour pass through a convolution layer with kernel size of 3*5*5 and stride of 1 respectively. After fusing the multiple modal feature maps extracted by the first convolution layer, they are input into the convolution layer with kernel size of 1*1*1 and stride of 1. Subsequently, the convolution layer with kernel size of 3*3*3 and stride of 1 and the convolution layer with kernel size of 3*3*3 and stride of 2 are input. Finally, the average pooling layer converts the feature map into a feature vector, and outputs the original output scores of the two types of labels after passing through the fully connected layer.

4. The radiation xerostomia prediction system based on multi-source channel fusion according to claim 1, characterized in that: The decision fusion module comprises: The stacking ensemble learning method is used to fuse the prediction results of the 3D Multi-GTVCNN model and the 3D Multi-PACNN model. First, the original output scores of the two models on the training set and the test set are obtained respectively. The original output scores of the two models on the training set are trained by the Naive Bayes algorithm. The Naive Bayes model will learn how to combine the prediction results of the 3D Multi-GTVCNN model and the 3D Multi-PACNN model. The final prediction result is obtained by using the trained Naive Bayes model to fuse the original output scores of the two models on the test set.

5. The radiation xerostomia depth prediction system based on multi-source channel fusion according to claim 4, characterized in that: The training of the 3D Multi-GTVCNN model uses the Adam optimizer for parameter learning, with an initial learning rate of 0.000005, a cosine annealing strategy to adjust the learning rate, a batch size of 1, an epoch of 150, an early stopping strategy to avoid model overfitting, and a patience of 10. The training of the 3D Multi-PACNN model uses the Adam optimizer for parameter learning, with an initial learning rate of 0.0001, a cosine annealing strategy to adjust the learning rate, a batch size of 4, an epoch of 150, an early stopping strategy to avoid model overfitting, and a patience of 10. The loss function uses the cross-entropy loss function, which evaluates the difference between the probability distribution output by the model and the true label by comparing the difference between the true probability distribution and the predicted probability distribution. For the binary classification task, the specific expression is: in, is the probability that the model predicts that the sample is a positive example, y is the sample label, if the sample is a positive example, the value is 1, otherwise the value is 0. In the embodiment of the present application, the positive example is radiation xerostomia after radiotherapy.

6. The radiation xerostomia depth prediction system based on multi-source channel fusion according to claim 1, characterized in that: The prediction module comprises: The true positive rate TP, false positive rate FP, true negative rate TN, and false negative rate FN corresponding to the model prediction are calculated respectively. The accuracy calculation unit is used to calculate the accuracy of the model F1 value in Sensitivity Specificity The ROC curve is generated based on the probability value of the classification result, and the corresponding AUC value is calculated to evaluate the performance of different models.

Images