Advanced nasopharynx cancer treatment effect prediction system based on deep learning

Abstract

The invention relates to the technical field of medical data processing, in particular to an advanced nasopharynx cancer treatment effect prediction system based on deep learning. According to the invention, a data acquisition module acquires multi-modal medical images and clinical data of a patient from a hospital information system; the data preprocessing module is used for cleaning, registering, normalizing and standardizing the data; the feature extraction module extracts image features by using a convolutional neural network and fuses the image features with clinical features to form a comprehensive feature vector; the model construction and training module constructs a deep learning model, and is adjusted and optimized through training; the treatment prediction module inputs the comprehensive feature vector to obtain a treatment effect prediction result; and the result evaluation and feedback module compares prediction and actual results, calculates indexes and collects and feeds back the improved model. The system comprehensively integrates data, accurately mines features, constructs an effective model, provides personalized prediction to assist clinical decision, continuously improves the model performance, and has important significance in improving the prediction accuracy of the treatment effect of advanced nasopharynx cancer, promoting personalized medical treatment and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of medical data processing, and in particular to a system for predicting the therapeutic efficacy of advanced nasopharyngeal carcinoma based on deep learning. Background Art

[0002] Nasopharyngeal carcinoma is a common malignant tumor of the head and neck, which seriously threatens the life and health of patients. Currently, radiotherapy is one of the main treatments for advanced nasopharyngeal carcinoma, but there are large individual differences in treatment effects. Some patients may experience tumor recurrence, metastasis or serious treatment complications, which affect the patient's quality of life and prognosis.

[0003] Traditional methods for predicting the efficacy of treatment of advanced nasopharyngeal carcinoma mainly rely on clinical experience and simple statistical analysis. Clinicians roughly judge the treatment effect based on the patient's clinical symptoms, pathological type, clinical stage and other factors, but this method is highly subjective and has limited accuracy. Although statistical analysis methods can use historical data for prediction to a certain extent, they can often only consider limited factors and have difficulty in handling complex nonlinear relationships. They are also insufficient in the fusion analysis of multimodal data.

[0004] With the continuous development of medical imaging technology, multimodal medical images such as CT, MRI, PET-CT, etc. are being used more and more widely in the diagnosis and treatment of nasopharyngeal carcinoma. These images can provide rich information about the morphology, location, metabolic activity, etc. of the tumor. However, how to effectively integrate and utilize these multimodal imaging data and clinical data for accurate efficacy prediction has become one of the current challenges. Therefore, a deep learning-based treatment efficacy prediction system for advanced nasopharyngeal carcinoma is proposed to address the above issues. Summary of the invention

[0005] The purpose of the present invention is to provide a deep learning-based treatment efficacy prediction system for advanced nasopharyngeal carcinoma to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The deep learning-based treatment efficacy prediction system for advanced nasopharyngeal carcinoma includes:

[0008] Data acquisition module: The data acquisition module collects multimodal medical imaging data and clinical data of patients with advanced nasopharyngeal carcinoma. The clinical data includes clinical diagnosis information and treatment process data.

[0009] Data preprocessing module: The data preprocessing module cleans the multimodal medical imaging data and clinical data to remove noise, missing values ​​and outliers; uses the image registration algorithm to register the multimodal medical imaging data to align the images in space; normalizes the multimodal medical imaging data and standardizes the clinical data;

[0010] Feature extraction module: The feature extraction module extracts medical image features from pre-processed multimodal medical image data, and uses a deep learning algorithm to automatically learn and extract features. The deep learning algorithm is a convolutional neural network. It extracts features from clinical diagnosis information and treatment process data, and converts them into feature vectors using statistical analysis methods and encoding methods according to the data type. It extracts clinical features from clinical diagnosis information and treatment process data, and fuses medical image features and clinical features to form a comprehensive feature vector for deep learning prediction model input.

[0011] Model building and training module: The model building and training module builds a deep learning prediction model; the model is trained using the collected and preprocessed multimodal medical imaging data and clinical data, and the stochastic gradient descent and its variant optimization algorithm are used to divide the data set into training set, validation set and test set. The model performance is evaluated through the validation set and the hyperparameters are adjusted to prevent overfitting;

[0012] Treatment prediction module: The treatment prediction module inputs the comprehensive feature vector into the trained deep learning prediction model to predict the efficacy. The deep learning prediction model calculates the treatment efficacy prediction results, which include tumor control probability, survival probability, and recurrence risk.

[0013] Result evaluation and feedback module: The result evaluation and feedback module regularly compares the predicted results with the actual treatment results of the patients and calculates the evaluation indicators; collects feedback from clinicians and patients, and optimizes and improves the deep learning prediction model based on the feedback information.

[0014] As a preferred solution, when a deep learning algorithm is used to automatically learn and extract features, the deep learning algorithm is a convolutional neural network. The convolutional neural network contains multiple convolutional layers and pooling layers. Assume that the input feature map of the convolutional layer is , the convolution kernel is , the bias is , then the calculation formula of the output feature map of the convolutional layer is: ;

[0015] in, represents the input channel index, represents the channel index of the output feature map, Represents the spatial position index, Indicates The set of input channels associated with each output channel, is the activation function, Represents the convolution operation; the pooling layer downsamples the feature map. Let the input feature map be , the pooling window size is , then the output of the maximum pooling is The calculation formula is: ;

[0016] in, and n are index variables used to traverse the elements within the pooling window.

[0017] As a preferred solution, when extracting clinical features from clinical diagnosis information and treatment process data, the categorical variables are converted into feature vectors using one-hot encoding. Suppose the categorical variables have If a sample belongs to the first category, categories, then its one-hot encoding vector is The first position is 1, and the rest of the positions are 0; the continuous variables are binned and then features are extracted, and they are converted into feature vectors using statistical analysis methods according to the data type.

[0018] As a preferred solution, when building a deep learning prediction model, a deep neural network and a recurrent neural network architecture are selected. The hidden layer of the deep neural network uses a fully connected layer and uses ReLU as the activation function. The input vector of the fully connected layer is , the weight matrix is , the bias is , then the output vector calculation formula of the fully connected layer is: , the output layer sets the corresponding number of nodes according to the prediction task and uses Sigmoid or Softmax function activation. For the binary classification task, the model output is set to , is the output of the fully connected layer, then the Sigmoid function calculates the predicted probability ; For multi-classification tasks, set it to the output layer The input value of the node, then the Softmax function calculates the probability ,in, is the number of categories.

[0019] As a preferred solution, the internal cell state of the recurrent neural network in the model building and training module is and hidden state The update formula is as follows: ; ; ; ; ; ;

[0020] in, is the time step The input vector is and are the corresponding weight matrices and bias vectors, is the Sigmoid function, ⊙ represents element-wise multiplication, , , They are input gate, forget gate and output gate respectively. is the candidate cell state, is the input vector to the weight matrix of the input gate, Hide the state for the last moment to the weight matrix of the input gate, is the time step The hidden state of is the bias vector of the input gate, is the input vector To the weight matrix of the forget gate, Hide the state for the last moment To the weight matrix of the forget gate, is the bias vector of the forget gate, is the input vector To the weight matrix of the output gate, Hide the state for the last moment To the weight matrix of the exit, is the bias vector of the output gate, is the input vector to the weight matrix of the candidate cell state, Hide the state for the last moment to the weight matrix of the candidate cell state, is the bias vector of the candidate cell state, is the hyperbolic tangent function, Forget gate The cell state at the previous moment Perform element-level multiplication to control the degree of retention of cell state information at the previous moment. Represents the input gate With candidate cell states Perform element-wise multiplication to determine how much the new information updates the current cell state. Output gate and Perform element-wise multiplication to control the ratio of cell state information output to the hidden state, while introducing nonlinear transformation.

[0021] As a preferred solution, when training the model using collected and preprocessed multimodal medical imaging data and clinical data, the stochastic gradient descent and its variant optimization algorithms are used to divide the data set into training set, validation set and test set. The performance of the deep learning prediction model is evaluated through the validation set and the hyperparameters are adjusted to prevent overfitting.

[0022] As a preferred solution, the evaluation indicators include accuracy, recall, F1 value, and area under the receiver operating characteristic curve. The accuracy is calculated by the following formula: ;

[0023] in, is a true positive, is a true negative, is a false positive. It is a false negative;

[0024] The recall is calculated by the following formula: ;

[0025] The F1 value is calculated by the following formula: ;

[0026] in, .

[0027] As a preferred solution, the image registration algorithm is based on the mutual information principle and resamples the registered images to unify the resolution.

[0028] It can be seen from the technical solutions provided by the present invention that the deep learning-based treatment efficacy prediction system for advanced nasopharyngeal carcinoma provided by the present invention has the following beneficial effects: through the collaborative work of various modules, it achieves significant beneficial effects in many aspects, provides strong support for the treatment of advanced nasopharyngeal carcinoma, and greatly promotes the progress of medical technology in this field, which is specifically reflected in:

[0029] 1. By setting up a data acquisition module that is closely connected with the hospital information system, it is possible to comprehensively and systematically obtain various types of data from patients with advanced nasopharyngeal carcinoma; this module can not only automatically collect multimodal medical imaging data, such as CT, MRI, PET-CT, etc., but also accurately obtain rich clinical data, including clinical diagnostic information (such as pathological type, clinical stage, treatment plan) and treatment process data (such as radiotherapy dose, chemotherapy drug dose); this comprehensive data acquisition method lays a solid information foundation for the entire prediction system, effectively avoiding prediction bias caused by missing or incomplete data; imaging data of different modalities present tumor characteristics from multiple dimensions, complementing clinical data, so that subsequent analysis and prediction can fully consider the patient's overall condition, significantly enhancing the adaptability and reliability of the prediction model to the complex conditions of different patients, and providing the possibility for accurate prediction of treatment efficacy;

[0030] 2. The data preprocessing module performs meticulous cleaning of the collected multimodal medical imaging data and clinical data. In terms of imaging data, advanced filtering algorithms are used to remove noise, such as median filtering and Gaussian filtering, which successfully achieves the goal of improving image clarity and accuracy, making the boundaries of the tumor area more clearly identifiable and the features more significant, creating favorable conditions for subsequent precise feature extraction;

[0031] 3. The feature extraction module uses advanced deep learning algorithms (such as convolutional neural networks) to automatically learn and extract features from pre-processed multimodal medical image data, thereby mining deep and complex feature patterns in images. The deep learning algorithm, with its powerful automatic feature learning ability, can accurately identify important information such as subtle lesion characteristics, texture changes, and subtle relationships with surrounding tissues of tumors. Compared with traditional manual feature extraction methods, this module provides more representative and accurate image features for efficacy prediction, greatly improving the prediction model's ability to depict tumor characteristics, thereby providing strong support for more accurate prediction of treatment efficacy. The module uses stochastic gradient descent and its The variant optimization algorithm trains the model and scientifically divides the data set into training set, validation set and test set. The validation set is used to evaluate the model performance and fine-tune the hyperparameters. This series of operations successfully prevents overfitting. The optimization algorithm can quickly and accurately adjust the model parameters according to the gradient information of the loss function, so that the model continuously approaches the optimal solution. At the same time, through real-time monitoring of the validation set and reasonable adjustment of hyperparameters, the model is guaranteed to have good generalization ability. This ensures that the model not only performs well on the training data, but also maintains accurate prediction ability when facing new patient data, effectively improving the practicality and stability of the model, so that it can play a reliable role in actual clinical applications.

[0032] 4. The treatment prediction module inputs the comprehensive feature vector into the trained deep learning prediction model, achieving the important effect of providing personalized treatment efficacy prediction results (including tumor control probability, survival probability, recurrence risk, etc.) for each patient with advanced nasopharyngeal carcinoma; these quantitative prediction results provide a key basis for clinicians to formulate personalized treatment plans, enabling doctors to comprehensively weigh the risks and benefits of treatment according to the patient's specific prediction situation, and thus choose the treatment strategy that best suits the patient. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a schematic diagram of the overall structure of the advanced nasopharyngeal carcinoma treatment efficacy prediction system based on deep learning of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0035] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0036] like Figure 1 As shown, an embodiment of the present invention provides a system for predicting the therapeutic efficacy of advanced nasopharyngeal carcinoma based on deep learning, including a data acquisition module, a data preprocessing module, a feature extraction module, a model building and training module, a treatment prediction module and a result evaluation and feedback module.

[0037] In this embodiment, the data acquisition module acquires multimodal medical imaging data and clinical data of patients with advanced nasopharyngeal carcinoma, where the clinical data includes clinical diagnosis information and treatment process data; specifically, the following operations are included:

[0038] 1. Multimodal medical imaging data acquisition

[0039] Connect with hospital imaging system:

[0040] Establish a secure connection with the hospital's PACS (Picture Archiving and Communication System) to ensure the stability and confidentiality of data transmission;

[0041] Verify the validity of the connection and test whether the patient's CT, MRI, PET-CT and other imaging data can be obtained normally;

[0042] Image data query and screening:

[0043] Query the relevant imaging examination records of the patient with advanced nasopharyngeal carcinoma in the PACS system according to the patient's unique identifier (such as medical record number, ID number, etc.);

[0044] Screen out imaging data that meets the requirements, such as CT, MRI, and PET-CT imaging data whose examination time is within a specific treatment stage and whose image quality meets the analysis standards (resolution, contrast, etc. meet certain requirements);

[0045] Data download and storage:

[0046] Download the filtered image data from the PACS system to the local database in a preset format (such as DICOM format);

[0047] Create indexes for imaging data, record patient information, examination time, imaging type and other metadata, and facilitate subsequent data management and retrieval;

[0048] 2. Collection of clinical diagnosis information

[0049] Connect with electronic medical record system:

[0050] Interface with the hospital's electronic medical record system to ensure access to information related to the patient's clinical diagnosis;

[0051] Perform authority verification to ensure the legitimacy of data acquisition;

[0052] Information extraction and organization:

[0053] Extract basic patient information from electronic medical records, such as name, age, gender, contact information, etc.;

[0054] Extract information related to the diagnosis of NPC, including pathological type (such as squamous cell carcinoma, adenocarcinoma and other specific types), clinical stage (such as TNM stage), diagnosis time, etc.;

[0055] Organize relevant information about the treatment plan, such as whether surgical treatment has been received, surgical method, radiotherapy plan (radiotherapy technology, total dose, fractionated dose, etc.), chemotherapy plan (chemotherapy drug type, dose, course of treatment, etc.);

[0056] Data format conversion and storage:

[0057] Convert the extracted clinical diagnosis information into a format that is compatible with subsequent data processing and model training requirements; for example, convert the pathology type in text format into a coded form (such as one-hot encoding or other categorical encoding methods);

[0058] Store the collated clinical diagnosis information in a local database and associate it with the patient's imaging data to ensure data integrity and consistency;

[0059] 3. Data collection during treatment

[0060] Treatment record acquisition:

[0061] Obtain detailed data on the patient's treatment process from the hospital's treatment information system (such as radiotherapy treatment record system, chemotherapy management system, etc.);

[0062] For radiotherapy, the actual execution parameters of each radiotherapy are obtained, such as the output dose of the radiotherapy equipment, irradiation time, irradiation field shape and size, treatment position, etc.

[0063] For chemotherapy, obtain data such as actual dosage, administration time, administration route, adverse reaction records, etc. of chemotherapy drugs;

[0064] Data integration and verification:

[0065] Integrate treatment process data obtained from different systems to ensure data consistency and accuracy;

[0066] Verify the rationality of the data, such as checking whether the radiotherapy dose is within the normal treatment range, whether the chemotherapy drug dose complies with the protocol, etc.;

[0067] Storage and Update:

[0068] The integrated and verified treatment process data is stored in a local database and associated with the patient's imaging data and clinical diagnosis information;

[0069] During the patient's treatment, the treatment process data is updated regularly to ensure the timeliness and completeness of the data;

[0070] 4. Data integrity and quality check

[0071] Sanity Check:

[0072] Regularly check whether the collected multimodal medical imaging data, clinical diagnosis information and treatment process data are complete; for example, ensure that each patient has corresponding CT, MRI and PET-CT images (if these examinations are performed during the treatment process), key fields in the clinical diagnosis information (such as pathological type, clinical stage) cannot be empty, and the treatment process data covers the main treatment links (radiotherapy, chemotherapy, etc.);

[0073] For missing data, communicate with relevant departments of the hospital (such as imaging department, radiotherapy department, chemotherapy department, medical record room, etc.) to find original records or collect additional data;

[0074] Quality Check:

[0075] Evaluate the quality of image data to check whether there are artifacts, blur, and other issues that affect analysis; for image data that does not meet the quality standards, determine whether it can be improved through image preprocessing methods (such as denoising, contrast enhancement, etc.), or whether it needs to be re-collected;

[0076] Conduct logical checks on clinical diagnosis information and treatment process data, such as whether the pathological type matches the treatment plan, whether the dose changes during the treatment process are reasonable, etc.; verify and correct data with logical errors;

[0077] 5. Manual data import (optional)

[0078] The interface provides:

[0079] Provide users with an interface for manually importing external data, such as allowing users to upload patient imaging data obtained from other medical institutions (must meet certain format requirements) or supplementary clinical information (such as special examination results conducted outside the hospital, etc.);

[0080] Format validation and conversion:

[0081] Verify the format of data manually imported by users to ensure that it is consistent with system requirements; if the format is incompatible, provide data format conversion tools or prompt users to perform corresponding conversions;

[0082] Data integration and storage:

[0083] Integrate manually imported data with other collected data and store them in the local database in the same storage structure and association method.

[0084] In this embodiment, the data preprocessing module cleans the multimodal medical imaging data and clinical data to remove noise, missing values ​​and outliers; registers the multimodal medical imaging data using an image registration algorithm to align the images in space; normalizes the multimodal medical imaging data and standardizes the clinical data; specifically includes the following steps:

[0085] 1. Data cleaning steps

[0086] Image data noise removal (for multimodal medical imaging data):

[0087] Noise detection: For CT, MRI, PET-CT and other imaging data, use appropriate noise detection algorithms, such as statistical methods based on pixel grayscale value distribution; calculate the difference in grayscale value between each pixel in the image and its neighboring pixels. If the difference exceeds the set threshold, the pixel may be a noise point;

[0088] Filtering:

[0089] If median filtering is used, for each pixel, select its neighborhood (such as The pixel values ​​in the window are sorted and the median is taken as the new value of the pixel after filtering; for example, for a The pixel values ​​in the neighborhood are 10, 20, 30, 15, 25, 35, 12, 22, 32. After sorting, they are 10, 12, 15, 20, 22, 25, 30, 32, 35. The median value is 22, so the value of the center pixel after filtering is 22.

[0090] If Gaussian filtering is used, according to the set standard deviation Calculate Gaussian kernel function ,in, is the pixel coordinate; then the original image is convolved with the Gaussian kernel function to obtain the filtered image;

[0091] Processing of missing values ​​in clinical data (for clinical data):

[0092] Missing value detection: traverse each field in the clinical data and count the number and distribution of missing values; for example, check whether there are empty or invalid values ​​in fields such as patient age, pathological type, and treatment dose;

[0093] Filling method selection:

[0094] For numeric fields (such as age and treatment dose), if the number of missing values ​​is small, mean filling can be used, that is, the average value of all non-missing values ​​in the field is calculated and filled into the missing value position; if there are many missing values ​​and the data distribution is uneven, median filling or interpolation methods based on data distribution characteristics (such as linear interpolation) can be considered;

[0095] For categorical fields (such as pathological type), if there are missing values, the most common category can be selected for filling according to the distribution of other non-missing values ​​of the field, or model prediction can be used (such as using other related fields to build a decision tree model to predict missing categorical values);

[0096] Outlier processing (for multimodal medical imaging data and clinical data):

[0097] Outlier detection (image data): For image data, outliers can be detected based on the statistical properties of the pixel values; for example, calculating the mean of the pixel values ​​in an image and standard deviation , the pixel value is ( The threshold is set based on experience, such as 3 or 5). Values ​​outside the range are considered outliers;

[0098] Outlier detection (clinical data): For numerical fields in clinical data, the mean and standard deviation are also calculated to detect outliers; for categorical fields, check whether there are values ​​that do not fit the predefined category range;

[0099] Correction or deletion of outliers: For outlier pixels in image data, corrections can be made based on their neighborhood pixel values, such as using mean replacement or median replacement; for outliers in clinical data, if they are obviously wrong (such as negative age), the record can be deleted directly; if they are possible measurement errors, corrections can be made based on professional knowledge and data distribution (such as adjusting the treatment dose that exceeds the reasonable range to within a reasonable range);

[0100] 2. Image registration steps

[0101] Feature point extraction (for multimodal medical imaging data):

[0102] Extract feature points from medical images of different modalities (such as CT and MRI); for CT images, their grayscale information and anatomical structure features can be used to extract feature points such as organ contours through edge detection algorithms (such as Canny edge detection); for MRI images, feature points are extracted based on the signal intensity differences of different tissues; for example, in brain images, edge points of structures such as the skull and ventricles can be extracted as feature points;

[0103] Similarity measure calculation:

[0104] A similarity measurement method based on mutual information is used; the mutual information between feature points of different modal images is calculated, and the mutual information reflects the degree of mutual dependence between two random variables; the larger the mutual information value, the higher the similarity between the two images; set For images and The mutual information of is calculated as:

[0105] in, It's an image The eigenvalue is And image The eigenvalue is The joint probability distribution of and Images and The marginal probability distribution of the eigenvalues ​​in ;

[0106] Transformation model estimation:

[0107] Based on the calculated similarity measure, an optimization algorithm (such as gradient descent method) is used to estimate the transformation model parameters that transform one image into another image space; common transformation models include rigid transformation (only translation and rotation), affine transformation (including translation, rotation, scaling and shearing), etc.; for example, for rigid transformation, three translation parameters and three rotation parameters need to be estimated; for affine transformation, more parameters need to be estimated to describe transformations such as scaling and shearing;

[0108] Image resampling and registration:

[0109] According to the estimated transformation model parameters, one of the images is resampled to align it with the other image in space. During the resampling process, the corresponding position of each pixel in the original image in the target image space is calculated according to the transformation model, and the pixel value at that position is determined by an interpolation method (such as bilinear interpolation). For example, if the coordinates of a pixel in the original image are , the coordinates in the target image after transformation are , using bilinear interpolation The new pixel value of the point is calculated based on the values ​​of the four nearest neighboring pixels.

[0110] 3. Normalization and standardization steps

[0111] Normalization of multimodal medical imaging data (for multimodal medical imaging data):

[0112] Determine the normalization range: Usually the pixel values ​​of the image are normalized to Interval or interval;

[0113] Calculate the normalization parameter: For Normalization of the interval, calculating the minimum pixel value in the image and maximum value ; Let the gray value of a pixel in the original image be , the normalized pixel value The calculation formula is If adopted Normalization of the interval can first map the pixel value to interval, and then through linear transformation Convert it to interval;

[0114] Clinical data standardization (for clinical data):

[0115] Calculate mean and standard deviation: For numeric fields in clinical data (such as age, treatment dose, etc.), calculate their mean and standard deviation ;

[0116] Standardized calculation: Assume that a value in the original clinical data is , the standardized value The calculation formula is ; This can make the clinical data have the characteristics of zero mean and unit standard deviation, which is convenient for subsequent data analysis and model processing; for categorical fields, standardization is generally not required, but encoding conversion (such as one-hot encoding) may be required to make it suitable for model input.

[0117] In this embodiment, the feature extraction module extracts medical image features from the preprocessed multimodal medical image data, and uses a deep learning algorithm to automatically learn and extract features, and the deep learning algorithm is a convolutional neural network; extracts features from clinical diagnosis information and treatment process data, and converts them into feature vectors according to the data type using statistical analysis methods and encoding methods; extracts clinical features from clinical diagnosis information and treatment process data, and fuses medical image features and clinical features to form a comprehensive feature vector for deep learning prediction model input; specifically includes the following steps:

[0118] 1. Steps of medical image feature extraction based on convolutional neural network

[0119] Network construction and initialization:

[0120] Construct a convolutional neural network architecture and determine the number of network layers, convolution kernel size, step size and other parameters; for example, you can use a structure that alternates multiple convolution layers and pooling layers, such as setting the convolution kernel size of the convolution layer to , the step size is 1, and the pooling layer uses The maximum pooling operation with a step size of 2;

[0121] Initialize the network's weights and bias parameters, either randomly or using a pre-trained model (if a relevant pre-trained model is available).

[0122] Image data input and forward propagation:

[0123] Input the preprocessed multimodal medical imaging data (such as normalized CT, MRI, and PET-CT images) into the convolutional neural network;

[0124] In the convolutional layer, for the input image feature map , convolution kernel and bias , the output feature map is calculated through convolution operation , the calculation formula is ,in, represents the input channel index, represents the channel index of the output feature map, Represents the spatial position index, represents the set of input channels associated with the jth output channel, For activation functions (such as ReLU function ,* represents convolution operation;

[0125] After the convolution layer, downsampling is performed through the pooling layer, such as the maximum pooling operation, for the input feature map , the pooling window size is , the output of the maximum pooling The calculation formula is:

[0126] Repeat the operations of the convolution layer and the pooling layer to gradually extract the high-level features of the image until the final output feature vector of the network is obtained;

[0127] Feature extraction and output:

[0128] The feature vector output by the last layer of the convolutional neural network is used as the medical image feature extracted from the medical image data. The feature vector can represent the tumor morphology, texture, function and other information in the image.

[0129] 2. Clinical data feature extraction steps

[0130] Categorical variable coding (for categorical variables in clinical diagnosis information):

[0131] For categorical variables in clinical diagnostic information (such as pathological type, treatment plan, etc.), determine the number of categories of the variable ;

[0132] Use one-hot encoding to convert categorical variables into feature vectors; if a sample belongs to categories, then its one-hot encoding vector is For example, if there are three pathological types: squamous cell carcinoma, adenocarcinoma, and undifferentiated carcinoma, the one-hot encoding vector for the squamous cell carcinoma sample is ;

[0133] Continuous variable feature extraction (for continuous variables in clinical data):

[0134] For continuous variables in clinical data (such as age, treatment dose, etc.), calculate their statistical features as part of the feature vector; for example, calculate the mean, standard deviation, maximum, minimum and other statistical values ​​of age, and form these values ​​into a feature vector; or after binning the continuous variable (such as equal-width binning, equal-frequency binning), count the number or proportion of samples in each bin as a feature;

[0135] Feature combination and arrangement:

[0136] Combine the encoded categorical variable feature vector and continuous variable feature vector to form a feature vector for clinical data; ensure that the order and correspondence of different types of features in the vector are correct so that they can be subsequently fused with medical imaging features;

[0137] 3. Feature fusion steps

[0138] Feature vector dimension adjustment (optional):

[0139] Check the dimensions of the medical image feature vector and the clinical feature vector. If the dimensions of the two do not match, they can be adjusted by appropriate methods. For example, if the dimension of the medical image feature vector is high, the dimension can be reduced to a dimension close to that of the clinical feature vector by using dimensionality reduction methods such as principal component analysis (PCA). If the dimension of the clinical feature vector is low, its dimension can be expanded to match that of the medical image feature vector by adding zero vectors or repeating certain eigenvalues.

[0140] Feature fusion operation:

[0141] The adjusted medical image feature vector and clinical feature vector are concatenated to form a comprehensive feature vector; for example, if the medical image feature vector is , the clinical feature vector is , then the comprehensive feature vector is ;

[0142] Comprehensive feature vector output:

[0143] The fused comprehensive feature vector is output as the input of the deep learning prediction model for subsequent efficacy prediction tasks.

[0144] In this embodiment, the model building and training module builds a deep learning prediction model; uses the collected and preprocessed multimodal medical imaging data and clinical data to train the model, uses the stochastic gradient descent and its variant optimization algorithm, divides the data set into a training set, a validation set, and a test set, evaluates the model performance through the validation set, and adjusts the hyperparameters to prevent overfitting; specifically includes the following operations:

[0145] 1. Steps to build a deep learning prediction model

[0146] Model architecture selection:

[0147] Determine the appropriate deep learning model architecture based on the characteristics and requirements of the prediction task; for example, for the task of predicting the therapeutic efficacy of advanced nasopharyngeal carcinoma that integrates image data and clinical data, you can choose an architecture that combines a convolutional neural network (CNN) with a fully connected layer, or consider using a recurrent neural network (RNN) and its variants (such as long short-term memory networks (LSTMs) and gated recurrent units (GRUs)) to process clinical data related to time series (such as dose change sequences during treatment). You can also explore the use of a Transformer-based architecture to better capture long-distance dependencies in the data;

[0148] Determine the input layer, hidden layer, and output layer structure of the model; the number of nodes in the input layer should match the dimension of the comprehensive feature vector (which integrates medical imaging features and clinical features); the number of hidden layers, the number of nodes, and the selection of activation functions need to be determined based on experiments and experience. Common activation functions include ReLU (RectifiedLinearUnit), Sigmoid, Tanh, etc.; the number of nodes in the output layer is determined according to the type of prediction task. For example, for a binary classification task of predicting tumor control probability and survival probability, the output layer can set a node and use the Sigmoid function for activation to obtain the probability value. For a multi-classification treatment effect evaluation task, the number of nodes in the output layer is the same as the number of categories, and the Softmax function is used for activation;

[0149] Model initialization:

[0150] Initialize the weights and bias parameters in the model. You can use random initialization methods, such as randomly generating uniformly distributed or normally distributed initial parameter values ​​within a certain range, or use the parameters of a pre-trained model for initialization (if a model trained on a similar task or dataset is available), and then fine-tune it on the dataset of this task.

[0151] 2. Model training steps

[0152] Dataset division:

[0153] The comprehensive feature vectors corresponding to the collected and preprocessed multimodal medical imaging data and clinical data and the corresponding real treatment efficacy labels (such as actual tumor control status, actual survival time, etc.) form a data set;

[0154] Randomly divide the data set into training set, validation set, and test set according to a certain ratio (such as 70%:20%:10% or other appropriate ratios); ensure that the training set, validation set, and test set have similar data distribution. For example, the patient's age, gender, disease stage and other characteristics have similar proportions in the three subsets to avoid bias caused by data division;

[0155] Set the training parameters:

[0156] Determine the number of training epochs, that is, the number of times the model performs a complete training on the entire training set. Initially, it can be set to a small value, such as 10-20 epochs, and then adjusted according to the training effect;

[0157] Select the optimization algorithm and its related parameters, such as using stochastic gradient descent (SGD) and its variants (such as Adagrad, Adadelta, Adam, etc.); for the Adam optimization algorithm, you need to set the learning rate (such as the initial learning rate is 0.001), (usually 0.9), (usually 0.999) and (usually 1e-8) and other parameters, which affect the update step size and direction of the model parameters;

[0158] Determine the loss function and select an appropriate loss function based on the type of prediction task; for binary classification tasks, you can use the binary cross entropy loss function (BinaryCross-EntropyLoss), the calculation formula is: ,in, is the true label (0 or 1), is the probability value predicted by the model; for multi-classification tasks, the multi-class cross entropy loss function (CategoricalCross-EntropyLoss) is used, and the calculation formula is ,in, is the one-hot encoding vector of the true label, is the probability vector predicted by the model, is the number of categories;

[0159] Model training process:

[0160] For each sample in the training set, its comprehensive feature vector is input into the model, and the model's predicted output is calculated through forward propagation;

[0161] Calculate the value of the loss function based on the predicted output and the true label;

[0162] Use the selected optimization algorithm (such as Adam) to back-propagate the gradient based on the value of the loss function and update the weights and bias parameters of the model; for example, the Adam optimization algorithm updates the parameters The formula is:

[0163] in, is the learning rate, and It is the correction value of the first-order moment estimate and the second-order moment estimate of the gradient, and its calculation involves , and historical gradient information;

[0164] Repeat the above steps to train all samples in the training set and complete one epoch of training;

[0165] After each round of training, the validation set is used to evaluate the performance of the model and calculate evaluation indicators (such as accuracy, recall, F1 value, loss function value, etc.);

[0166] 3. Hyperparameter adjustment and steps to prevent overfitting

[0167] Hyperparameter Tuning:

[0168] Adjust the model's hyperparameters based on the evaluation results of the validation set. Common hyperparameter adjustment methods include GridSearch, RandomSearch, etc. For example, for the learning rate, you can try For the number of hidden layers, you can choose Try different values ​​within the same range, and select the best hyperparameter combination by comparing the performance indicators of the validation set under different hyperparameter combinations;

[0169] If the model is found to be overfitting or underfitting on the validation set, the model complexity can be adjusted accordingly; for example, if it is overfitting, the number of hidden layers can be reduced and the regularization strength (such as L1 regularization, L2 regularization) can be increased; if it is underfitting, the number of hidden layers can be increased, the regularization strength can be reduced, or the number of training rounds can be increased.

[0170] Measures to prevent overfitting:

[0171] Regularization method:

[0172] L1 regularization: Add L1 regularization term to the loss function ,in, is the regularization coefficient, is the model parameter; L1 regularization can make the model parameters sparse and reduce the influence of unimportant features;

[0173] L2 regularization: Add L2 regularization term to the loss function , L2 regularization can limit the size of model parameters and prevent the model from being too complex;

[0174] Early Stopping: During the training process, monitor the performance indicators of the validation set (such as loss function value or accuracy); if the performance of the validation set does not improve significantly (or even decreases) in several consecutive rounds of training (such as 5-10 rounds), stop training early to prevent the model from overfitting the training data;

[0175] Data enhancement (optional, for image data): If there is multimodal medical image data, data enhancement technology can be used to increase the diversity of training data, such as rotating, flipping, translating, scaling, etc., to generate new training samples. However, it should be noted that data enhancement operations should conform to the actual physical meaning and diagnostic requirements of medical images to avoid introducing unreasonable changes.

[0176] Ensemble learning (optional): training multiple different models (such as by changing initialization parameters or using different data subsets for training), and then combining the prediction results of these models (such as voting method, weighted average method, etc.) can improve the generalization ability of the model and reduce the risk of overfitting; however, ensemble learning will increase computing costs and training time, so it needs to be used according to actual conditions.

[0177] 4. Model evaluation and selection steps

[0178] Final model evaluation:

[0179] After completing model training and hyperparameter adjustment, use the test set to evaluate the final model; calculate the accuracy, recall, F1 value, area under the receiver operating characteristic curve (ROC curve) (AUC) and other evaluation indicators on the test set to comprehensively evaluate the performance of the model; for example, the accuracy calculation formula is:

[0180] in, is a true positive, is a true negative, is a false positive. It is a false negative;

[0181] The larger the AUC value, the better the model performance. AUC measures the model's ability to distinguish between positive and negative examples by calculating the area under the ROC curve.

[0182] Model selection and saving:

[0183] According to the evaluation results of the test set, the model with the best performance is selected as the final deep learning prediction model;

[0184] Save the selected model to the local file system or model management platform for subsequent loading and use in the treatment prediction module. The saved model should contain the model's structure, weights, biases, and related parameter information used in the training process to ensure that the model's predictive ability can be accurately reproduced.

[0185] In this embodiment, the treatment prediction module inputs the comprehensive feature vector into the trained deep learning prediction model to predict the therapeutic effect. The deep learning prediction model calculates the therapeutic efficacy prediction result. The therapeutic efficacy prediction result includes the tumor control probability, the survival probability, and the recurrence risk. Specifically, the following operations are included:

[0186] Data preparation and input:

[0187] Obtain the comprehensive feature vector of the patient to be predicted after data collection, preprocessing and feature extraction; ensure that the format and dimension of the vector are consistent with the input requirements of the trained deep learning prediction model;

[0188] Input the comprehensive feature vector into the trained deep learning prediction model;

[0189] Model forward propagation calculation:

[0190] In the deep learning prediction model, the comprehensive feature vector is calculated through the input layer and hidden layer in turn; for the architecture based on the combination of convolutional neural network (CNN) and fully connected layer, the input vector is first multiplied by the weight matrix and biased in the fully connected layer, and then nonlinearly transformed by the activation function (such as ReLU). The calculation formula is ,in, is the input vector, is the weight matrix, is the bias, is the activation function, is the hidden layer output;

[0191] The output of the hidden layer is passed to the next layer, and the above calculation process is repeated until it reaches the output layer. For the binary classification task of predicting tumor control probability and survival probability, the output layer is activated using the Sigmoid function, and the model output is the predicted probability value, calculated as follows: ,in, is the input value output by the fully connected layer to the output layer; for multi-classification treatment effect evaluation tasks, the output layer uses the Softmax function for activation to calculate the probability of each category ,in, The output layer The input value of the node, is the number of categories;

[0192] Treatment efficacy prediction results generation:

[0193] According to the calculation results of the model output layer, the predicted values ​​such as tumor control probability and survival probability are obtained; for example, if the model output is a value between 0 and 1, it indicates the prediction of tumor control probability or survival probability;

[0194] For the prediction of recurrence risk, further processing can be performed based on the probability value output by the model; for example, a threshold (such as 0.5) is set, and when the predicted tumor control probability is lower than the threshold, the recurrence risk is considered to be high; or based on the statistical analysis of a large amount of patient data, the predicted probability is mapped to the recurrence risk level (such as low risk, medium risk, high risk);

[0195] Result visualization and output (optional):

[0196] Visualize the treatment efficacy prediction results so that clinicians can understand and analyze them more intuitively; for example, display the tumor control probability and the trend of survival probability over time in a chart (if the model can provide time-related prediction information), or use color coding to indicate the recurrence risk level;

[0197] The prediction results are output to the clinician's work interface or system, and relevant explanatory information can be provided, such as the main features on which the model makes the prediction, to help doctors better refer to the prediction results to make treatment decisions.

[0198] In this embodiment, the result evaluation and feedback module regularly compares the predicted results with the actual treatment results of the patients and calculates the evaluation index; collects feedback from clinicians and patients, and optimizes and improves the deep learning prediction model based on the feedback information; specifically includes the following operations:

[0199] Data collection and compilation:

[0200] Obtain prediction results regularly: Obtain the treatment efficacy prediction results of patients who have completed treatment from the treatment prediction module according to the set time period (such as monthly, quarterly, etc.), including prediction values ​​such as tumor control probability, survival probability, recurrence risk, and corresponding patient identification information (such as medical record number);

[0201] Collect actual treatment results: collect the actual treatment results data of these patients from the hospital's clinical information system (such as electronic medical record system, follow-up system, etc.); for tumor control, imaging examinations (such as CT, MRI, etc.) and clinical evaluation can be used to determine whether there is recurrence; survival time can be obtained from the patient's discharge record, follow-up record or death registration information; recurrence risk can be qualitatively or quantitatively evaluated based on the actual recurrence situation (such as recurrence time, recurrence degree, etc.);

[0202] Data matching and integration: Match and integrate the predicted result data with the actual treatment result data according to the patient identifier to ensure that the predicted information and actual information of each patient correspond one-to-one, forming a data set for evaluation;

[0203] Evaluation indicator calculation:

[0204] Accuracy calculation: For two-class evaluation tasks such as tumor control and survival, calculate the accuracy. (TruePositive) is true positive (predicted to be positive and actually positive, such as predicted tumor control and actually no recurrence), (TrueNegative) is true negative (predicted to be negative and actually negative, such as predicting that the tumor is not under control and actually has recurred), (FalsePositive) is a false positive (predicted to be positive but actually negative), (FalseNegative) is a false negative (predicted to be negative but actually positive); the accuracy formula is as follows:

[0205] Recall calculation: The recall formula is as follows:

[0206] It is used to measure the ability of the model to correctly predict positive samples, that is, the proportion of samples that are actually positive that are predicted to be positive by the model;

[0207] The F1 value calculation formula is as follows:

[0208] in:

[0209] The F1 value takes into account both precision and recall, and can more comprehensively evaluate model performance;

[0210] Calculation of other evaluation indicators (optional): For example, the area under the receiver operating characteristic curve (ROC curve) (AUC) is calculated. The larger the AUC value, the stronger the model's ability to distinguish between positive and negative examples. For survival analysis tasks, the consistency index (C-index) of the survival curve can be calculated to evaluate the consistency between the model's predicted survival time and the actual survival time.

[0211] Result analysis and visualization:

[0212] Indicator analysis: Analyze the calculated evaluation indicators to understand the performance of the model in different aspects. For example, if the accuracy is high but the recall is low, it may mean that the model performs well in avoiding false positives, but there are many false negatives. Further analysis is needed to find out the reasons, such as whether the data imbalance causes the model to have insufficient prediction ability for minority categories.

[0213] Visualization: The evaluation results are displayed in the form of graphs, such as ROC curves and precision-recall curves, to intuitively present the model performance. At the same time, the evaluation results can be stratified and analyzed according to different patient characteristics (such as age, gender, clinical stage, etc.) to view the performance differences of the model in different subgroups.

[0214] Feedback collection:

[0215] Clinician feedback: Collect clinicians' opinions on prediction results by regularly organizing meetings with clinicians, online questionnaires, or setting up feedback portals in the systems used by doctors. Clinicians can provide feedback on the usefulness of prediction results in actual clinical decision-making, whether the accuracy meets expectations, whether the model output is easy to understand, and suggestions for model improvement, such as which clinical factors they would like to add and how to improve the presentation of prediction results.

[0216] Patient feedback (optional): Under the premise of complying with ethical and privacy regulations, collect patients' feelings and opinions on the prediction results through patient follow-up surveys and other means; patients can provide feedback on whether the prediction results have an impact on their psychology, whether they are helpful for communication and expectation management during the treatment process, etc.;

[0217] Model optimization and improvement:

[0218] Data level improvements:

[0219] If data quality issues are found (such as missing data, incorrect labeling, etc.), optimize the data collection and preprocessing process based on the feedback information to ensure the accuracy and completeness of the data;

[0220] If the evaluation results show that the model performs poorly in certain patient groups, it may be due to uneven data distribution. Consider using data augmentation techniques (such as oversampling minority class samples, undersampling majority class samples, or generating synthetic samples) or collecting more representative samples to improve data distribution and enhance the generalization ability of the model.

[0221] Model architecture adjustment:

[0222] Based on feedback and evaluation results, if the model has difficulty processing specific types of data (such as certain features in imaging data, complex relationships in clinical data), consider adjusting the model architecture; for example, adding convolutional layers in the convolutional neural network or adjusting the size of the convolutional kernel to better extract imaging features; for processing clinical time series data, try replacing it with a more suitable recurrent neural network variant or introducing an attention mechanism to enhance the model's focus on key information;

[0223] If the model is too complex or simple, resulting in overfitting or underfitting problems, adjust the complexity of the model appropriately, such as increasing or decreasing the number of hidden layers, adjusting the number of nodes, or changing the activation function;

[0224] Model training optimization:

[0225] Adjust the hyperparameters of model training based on evaluation indicators and feedback information; for example, if the model converges too slowly or becomes unstable, adjust the learning rate (such as using an adaptive learning rate strategy), optimize the algorithm (such as trying other stochastic gradient descent variants), or increase the number of training rounds; if the model is found to be overfitting, increase the regularization strength (such as adjusting the L1 or L2 regularization coefficient) or adopt an early stopping training strategy (based on the performance monitoring of the validation set);

[0226] Consider using ensemble learning methods to train multiple different models (e.g., through different initializations or data subsets), and then combine their prediction results (e.g., voting method, weighted average method, etc.) to improve the stability and accuracy of the model;

[0227] Retraining and validation:

[0228] Retrain the deep learning prediction model using the improved dataset, model architecture, and training parameters;

[0229] During the retraining process, the validation set is used to monitor the model performance to ensure that the model improves in the expected direction during the optimization process; after the training is completed, the model is evaluated again using the test set to compare the performance changes of the model before and after optimization to verify the effectiveness of the optimization measures; if the performance has not been significantly improved or new problems arise, continue to analyze the causes and make adjustments until the model performance meets the requirements of clinical applications.

[0230] In one embodiment, when the recurrent neural network in the model building and training module processes sequence data, its internal cell state and hidden state The update formula is as follows: ; ; ; ; ; ;

[0231] in, is the time step The input vector is and are the corresponding weight matrices and bias vectors, is the Sigmoid function, ⊙ represents element-wise multiplication, , , They are input gate, forget gate and output gate respectively. is the candidate cell state, is the input vector to the weight matrix of the input gate, Hide the state for the last moment to the weight matrix of the input gate, is the time step The hidden state of is the bias vector of the input gate, is the input vector To the weight matrix of the forget gate, Hide the state for the last moment To the weight matrix of the forget gate, is the bias vector of the forget gate, is the input vector To the weight matrix of the output gate, Hide the state for the last moment To the weight matrix of the exit, is the bias vector of the output gate, is the input vector to the weight matrix of the candidate cell state, Hide the state for the last moment to the weight matrix of the candidate cell state, is the bias vector of the candidate cell state, is the hyperbolic tangent function, Forget gate The cell state at the previous moment Perform element-level multiplication to control the degree of retention of cell state information at the previous moment. Represents the input gate With candidate cell states Perform element-wise multiplication to determine how much the new information updates the current cell state. Output gate and Perform element-wise multiplication to control the ratio of cell state information output to the hidden state, while introducing nonlinear transformation.

[0232] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. Advanced nasopharyngeal carcinoma treatment efficacy prediction system based on deep learning, Features: include: Data acquisition module; The data acquisition module acquires multimodal medical imaging data and clinical data of patients with advanced nasopharyngeal carcinoma, wherein the clinical data includes clinical diagnosis information and treatment process data; Data preprocessing module: The data preprocessing module cleans the multimodal medical imaging data and clinical data to remove noise, missing values ​​and outliers; and registers the multimodal medical imaging data using an image registration algorithm to align the images in space; Normalize multimodal medical imaging data and standardize clinical data; Feature extraction module; The feature extraction module extracts medical image features from the pre-processed multimodal medical image data, and automatically learns and extracts features using a deep learning algorithm, where the deep learning algorithm is a convolutional neural network; extracts features from clinical diagnosis information and treatment process data, and converts them into feature vectors using statistical analysis methods and encoding methods according to the data type; extracts clinical features from the clinical diagnosis information and treatment process data, and fuses the medical image features and clinical features to form a comprehensive feature vector for input into a deep learning prediction model; Model building and training module; the model building and training module builds a deep learning prediction model; uses the collected and pre-processed multimodal medical imaging data and clinical data to train the model, uses the stochastic gradient descent and its variant optimization algorithm, divides the data set into a training set, a validation set and a test set, evaluates the model performance through the validation set and adjusts the hyperparameters to prevent overfitting; Treatment prediction module: The treatment prediction module inputs the comprehensive feature vector into the trained deep learning prediction model to predict the therapeutic effect. The deep learning prediction model calculates the treatment efficacy prediction result. The treatment efficacy prediction result includes the tumor control probability, survival probability, and recurrence risk. Result evaluation and feedback module: The result evaluation and feedback module regularly compares the predicted results with the actual treatment results of the patients, calculates the evaluation indicators, collects feedback from clinicians and patients, and optimizes and improves the deep learning prediction model based on the feedback information.

2. The deep learning-based treatment efficacy prediction system for advanced nasopharyngeal carcinoma according to claim 1, Features: When the deep learning algorithm is used to automatically learn and extract features, the deep learning algorithm is a convolutional neural network, which includes multiple convolutional layers and pooling layers. Assume that the input feature map of the convolutional layer is , the convolution kernel is , the bias is , then the calculation formula of the output feature map of the convolutional layer is: ； in, represents the input channel index, represents the channel index of the output feature map, Represents the spatial position index, Indicates The set of input channels associated with each output channel, is the activation function, Represents the convolution operation; the pooling layer downsamples the feature map. Let the input feature map be , the pooling window size is , then the output of the maximum pooling is The calculation formula is: ； in, and n are index variables used to traverse the elements within the pooling window.

3. The deep learning-based treatment efficacy prediction system for advanced nasopharyngeal carcinoma according to claim 2, Features: When extracting clinical features from clinical diagnosis information and treatment process data, the categorical variables are converted into feature vectors using a one-hot encoding method. Suppose the categorical variables have If a sample belongs to the first category, categories, then its one-hot encoding vector is The first position is 1, and the rest of the positions are 0; the continuous variables are binned and then features are extracted, and they are converted into feature vectors using statistical analysis methods according to the data type.

4. The system for predicting the therapeutic efficacy of advanced nasopharyngeal carcinoma based on deep learning according to claim 3, Features: When constructing the deep learning prediction model, a deep neural network and a recurrent neural network architecture are selected. The hidden layer of the deep neural network adopts a fully connected layer and uses ReLU as the activation function. The input vector of the fully connected layer is , the weight matrix is , the bias is , then the output vector calculation formula of the fully connected layer is: , the output layer sets the corresponding number of nodes according to the prediction task and uses Sigmoid or Softmax function activation. For the binary classification task, the model output is set to , is the output of the fully connected layer, then the Sigmoid function calculates the predicted probability ; For multi-classification tasks, set the output layer The input value of the node, then the Softmax function calculates the probability ,in, is the number of categories.

5. The system for predicting the therapeutic efficacy of advanced nasopharyngeal carcinoma based on deep learning according to claim 4, Features: When the recurrent neural network in the model building and training module processes sequence data, its internal cell state and hidden state The update formula is as follows: ； ； ； ； ； ； in, is the time step The input vector is and are the corresponding weight matrices and bias vectors, is the Sigmoid function, ⊙ represents element-wise multiplication, , , They are input gate, forget gate and output gate respectively. is the candidate cell state, is the input vector to the weight matrix of the input gate, Hide the state for the last moment to the weight matrix of the input gate, is the time step The hidden state of is the bias vector of the input gate, is the input vector To the weight matrix of the forget gate, Hide the state for the last moment To the weight matrix of the forget gate, is the bias vector of the forget gate, is the input vector To the weight matrix of the output gate, Hide the state for the last moment To the weight matrix of the exit, is the bias vector of the output gate, is the input vector to the weight matrix of the candidate cell state, Hide the state for the last moment to the weight matrix of the candidate cell state, is the bias vector of the candidate cell state, is the hyperbolic tangent function, Forget gate The cell state at the previous moment Perform element-level multiplication to control the degree of retention of cell state information at the previous moment. Represents the input gate With candidate cell states Perform element-wise multiplication to determine how much the new information updates the current cell state. Output gate and Perform element-wise multiplication to control the ratio of cell state information output to the hidden state, while introducing nonlinear transformation.

6. The system for predicting the therapeutic efficacy of advanced nasopharyngeal carcinoma based on deep learning according to claim 5, Features: When the model is trained using the collected and preprocessed multimodal medical imaging data and clinical data, the stochastic gradient descent and its variant optimization algorithms are used to divide the data set into a training set, a validation set, and a test set. The performance of the deep learning prediction model is evaluated through the validation set and the hyperparameters are adjusted to prevent overfitting.

7. The system for predicting the therapeutic efficacy of advanced nasopharyngeal carcinoma based on deep learning according to claim 1, Features: The evaluation indicators include accuracy, recall, F1 value, and area under the receiver operating characteristic curve. The accuracy is calculated by the following formula: ； in, is a true positive, is a true negative, is a false positive. It is a false negative; The recall is calculated by the following formula: ； The F1 value is calculated by the following formula: ； in, .

8. The deep learning-based treatment efficacy prediction system for advanced nasopharyngeal carcinoma according to claim 1, Features: The image registration algorithm is based on the mutual information principle and resamples the registered images to unify the resolution.

Images