Prostate cancer medical intelligent prediction system based on deep learning

Abstract

The invention relates to the technical field of cancer detection. The invention aims to provide a prostate cancer medical intelligent prediction system based on deep learning. The prostate cancer medical intelligent prediction system comprises a Gleason score prediction system and a csPCA risk prediction system, the csPCA risk prediction system can perform comprehensive prediction and evaluation in combination with clinical data and a data set obtained by the Gleason score prediction system. According to the method, the deep learning technology is applied to the Gleason scoring prediction of the prostate cancer, so that the objectivity and consistency of the scoring process are remarkably improved, the pathological image can be automatically processed, the influence of human factors on the scoring process is avoided, and the scoring objectivity is improved. Regression loss and classification loss are innovatively combined, and an MSE loss function commonly used by a regression task and a Focal loss function commonly used by a classification task are combined to serve as a loss function of network training.

Description

Technical Field

[0001] The present invention relates to the technical field of cancer detection, and in particular to a medical intelligent prediction system for prostate cancer based on deep learning. Background Art

[0002] Currently, prostate cancer detection methods mainly include:

[0003] (1) Serum biomarker prostate-specific antigen (PSA) detection method:

[0004] PSA is currently the most commonly used prostate cancer screening indicator. However, due to the problems of false positives and false negatives, other biomarkers (such as fPSA / tPSA ratio, PSAD, etc.) are usually combined in clinical practice to improve the specificity of diagnosis.

[0005] (2) Imaging methods, such as:

[0006] 1) Multiparametric magnetic resonance imaging (mpMRI) combined with PI-RADS scoring system:

[0007] The Prostate Imaging Reporting and Data System (PI-RADS) is a standardized scoring system for assessing prostate lesions, especially the possibility of prostate cancer. The system was developed and published by the American College of Radiology, the European Society of Urogenital Radiology, and the AdMeTech Foundation. It aims to provide clinicians with a quantitative assessment of the risk of prostate cancer by comprehensively analyzing the imaging characteristics of prostate magnetic resonance imaging (MRI). The PI-RADS scoring system is mainly divided into five levels, each representing a different risk of prostate cancer. Although the PI-RADS scoring system has important application value in the diagnosis of prostate cancer, it also has certain limitations. Due to the complex and diverse imaging manifestations of prostate MRI, different doctors may have differences in reading the film, which may lead to inconsistent scoring results. In addition, some special types of prostate cancer may have atypical manifestations on MRI and are difficult to be accurately assessed. Therefore, when applying the PI-RADS scoring system, doctors need to comprehensively consider multiple factors such as the patient's clinical manifestations, serum biomarkers, other imaging examination results, and pathological information to make a more comprehensive and accurate diagnosis.

[0008] 2) PSMA-PET

[0009] This is a new type of molecular imaging technology that uses prostate-specific membrane antigen (PSMA) as a target. It can more accurately detect prostate cancer and its micrometastases, and improve the accuracy of prostate cancer risk assessment.

[0010] (3) Prostate cancer risk calculators, such as:

[0011] 1) D'Amico Prostate Cancer Risk Calculator: This is a risk assessment tool based on simple indicators such as PSA level, Gleason score and clinical stage to assess the risk of recurrence of prostate cancer. It helps doctors develop more personalized treatment plans and assess the prognosis of patients.

[0012] 2) PCPT risk calculator: It is a tool that combines multiple factors to assess an individual's risk of developing prostate cancer. It usually considers multiple factors such as age, race, serum PSA (prostate-specific antigen) level, biopsy history, and family history, and provides an individual with a quantitative assessment of prostate cancer risk through complex statistical models and algorithms. It is better than a simple PSA test in predicting prostate cancer and can effectively distinguish between low-grade and high-grade prostate cancer.

[0013] 3) ERSPC Risk Calculator: ERSPC (European Randomized Study of Screening for Prostate Cancer) is a large-scale prostate cancer screening project involving male participants from multiple European countries. Based on the data from this study, the ERSPC Prostate Cancer Risk Calculator was developed. It includes parameters such as AUA symptom score, PSA (prostate-specific antigen), TRUS (transrectal ultrasound) and DRE (digital rectal examination). It can be used to judge the situation of prostate cancer and provide a reference for clinical decision-making.

[0014] 4) Sunnybrook Risk Calculator: includes parameters such as PSA, fPSA (free prostate-specific antigen), age, race, family history, International Prostate Symptom Score and DRE. It can be used to predict the risk of prostate cancer and provide a basis for personalized screening and treatment plans.

[0015] 5) Prostate cancer risk calculator released by Guangzhou First People's Hospital: Developed based on prostate cancer diagnosis data of Chinese men, it takes into account parameters such as age, serum PSA, prostate volume, and rectal examination results. It has a high accuracy rate and can be used as an important reference for clinical decisions on whether a prostate puncture biopsy is needed.

[0016] (4) Genetic testing products for accurate diagnosis and treatment of prostate cancer;

[0017] Based on the progress of sequencing technology, these products provide patients with personalized risk assessment and treatment options by detecting gene mutations related to the occurrence and development of prostate cancer. For example, the series of gene detection products for accurate diagnosis and treatment of prostate cancer designed and developed by Rendong Medical can comprehensively cover prostate cancer-related genes and evaluate prognosis, medication, typing, genetics and other diagnosis and treatment-related issues.

[0018] Although there are many methods for detecting prostate cancer, the early symptoms of prostate cancer are not obvious, resulting in many patients being in the advanced stage or csPCa stage when diagnosed, which makes treatment difficult. The Gleason grading of prostate cancer is crucial for risk stratification and treatment selection of patients, but the grading process is complex and subjective, and even pathologists often have large disagreements.

[0019] In recent years, with the rapid development of deep learning and artificial intelligence, as a powerful machine learning technology, it has performed well in image recognition, classification and regression. Its feasibility as a prerequisite for prostate cancer prediction has become more mature, and the industry has been trying to improve the overall accuracy and objectivity of prostate pre- / detection through various intelligent methods. However, up to now, there is still no real method that is highly operational and has high accuracy in prediction and evaluation.

[0020] Therefore, it is particularly important to develop a system and prediction method that can reduce observer differences and improve scoring accuracy and repeatability, which can provide new ideas for the prediction of Gleason scores for prostate cancer. Summary of the invention

[0021] The purpose of the present invention is to provide a medical intelligent prediction system for prostate cancer based on deep learning, which can automatically and intelligently predict the patient's Gleason score based on deep learning regression model and clinical data, and significantly improve the objectivity and consistency of the Gleason score, ensure the standardization and repeatability of the entire scoring process, and provide a guarantee for improving the efficiency and quality of medical services.

[0022] To achieve the above-mentioned purpose of the invention, the technical solution adopted by the present invention is: a medical intelligent prediction system for prostate cancer based on deep learning, including a Gleason score prediction system and a csPCA risk prediction system; the csPCA risk prediction system can combine clinical data and the data set obtained by the Gleason score prediction system for comprehensive prediction and evaluation;

[0023] The working steps of the Gleason score prediction system are as follows:

[0024] S1.1. Training phase

[0025] S1.1.1, Data preparation: Collect a dataset containing images of prostate cancer patients and their corresponding Gleason scores; images of prostate cancer patients include DWI images, ADC images, and T2 images;

[0026] S1.1.2, Image preprocessing:

[0027] Perform bias field correction and registration on images;

[0028] S1.1.3, divide the training set, validation set and test set into the following proportions: 70% training set, 10% validation set and 20% test set.

[0029] S1.1.4, Model selection and hyperparameter setting: Select a deep learning neural network model suitable for regression tasks; adjust the model structure and hyperparameters according to task requirements, and modify the output layer to ensure that the output layer should have only one neuron; the hyperparameters include at least: resolution, cropping size, maximum number of iterations, initial learning rate, decay rate, optimizer, and batch size;

[0030] S1.1.5, loss function selection: Combined with the MSE loss function L commonly used in regression tasks MSE The Focal loss function L commonly used in classification tasks Focal According to formula (1), select an appropriate loss function L loss To measure the difference between the model output and the true label;

[0031] L loss = ωL MSE + (1-ω) L focal (1)

[0032] In formula (1), L loss Represents the loss function of network training, L MSE represents the mean square error loss, L focal Represents focal loss. ω is the weight balance parameter between mean square error loss and focal loss, and its value range is 0-1. When ω is 0, the loss function of network training is focal loss; when ω is 1, the loss function of network training is mean square error loss; when ω is other values ​​between 0-1, the loss function of network training is the weighted sum of mean square error loss and focal loss;

[0033] S1.1.6, data reading: The number of samples read each time is determined by the batch size of the hyperparameter, which is the number of samples processed simultaneously during each iteration of training; a batch size of samples is randomly selected from the training data set, and the images of these samples preprocessed by S1.1.2 are read into the model;

[0034] S1.1.7, Cropping and resampling: According to the prostate cancer annotation results, the position of the prostate cancer in the whole image is obtained. Within the range of the prostate cancer, image blocks of fixed size and resolution are randomly sampled. The resolution and size are determined by the hyperparameters spacing and crop_size;

[0035] S1.1.8, data preprocessing: perform data enhancement operations on the data; then use the patient image as a unit, use adaptive normalization processing to remove voxels with MR image intensity values ​​higher than 99% and lower than 1%, and use maximum and minimum normalization processing to unify all image intensity values ​​between 0 and 1;

[0036] S1.1.9, forward propagation: a batch_size of preprocessed images are input into the model. Through the forward propagation operation, the image will pass through each neuron in the network layer by layer to predict the Gleason score;

[0037] S1.1.10, Loss calculation: Calculate the loss between the network predicted Gleason score and the true Gleason score according to the selected loss function and calculate the gradient, and use the back propagation algorithm to update the model parameters;

[0038] S1.1.11, deep learning network model training: repeat S1.1.6 to S1.1.10, traverse the entire training set data once to complete one iteration of training; then repeat this step until the maximum number of iterations is reached, that is, the entire model training process is completed;

[0039] S1.1.12, Model evaluation and tuning: Evaluate model performance on the validation set, obtain mean absolute error, mean square error, R-square coefficient, Pearson coefficient, and select the optimal model;

[0040] S1.2. Reasoning stage

[0041] S1.2.1, Image preprocessing: The method is the same as S1.1.2, including bias field correction and registration.

[0042] S1.2.2, data reading: read the three image sequences of the test sample: DWI image, ADC image and T2 image, as well as the prostate cancer lesion marking mask of the T2 image into the model; according to the prostate cancer lesion marking mask, its position in the whole image can be obtained, including the center point coordinates and length, width and height information;

[0043] S1.2.3, Cropping and resampling: Taking the coordinates of the center point of the prostate cancer lesion marking mask in the original image as the center, sample image blocks of fixed size and resolution. The resolution and size are determined by the hyperparameters resolution and cropping size during training;

[0044] S1.2.4, data preprocessing: preprocess the read image blocks according to the image normalization method set in the training phase;

[0045] S1.2.5, forward propagation: load the trained deep learning regression model, input the preprocessed image blocks into the model for forward propagation, and calculate the Gleason score prediction value; select the integer closest to the prediction value as the final Gleason score prediction result;

[0046] S1.2.6, Calculation of evaluation indicators: Calculate the performance indicators of the model on the test set based on the difference between the predicted results and the true labels, and obtain the mean absolute error, mean square error, R square coefficient, and Pearson coefficient.

[0047] S1.2.7, Result Visualization: Generate visualization tools based on test results to better understand the performance of the model on the regression task;

[0048] The working steps of the csPCA risk prediction system are as follows:

[0049] S2.1 Training Phase

[0050] S2.1.1, data preparation: In addition to the data set containing three-sequence DWI images, ADC images and T2 images of prostate cancer patients and the corresponding Gleason scores collected in step S1 above, clinical data needs to be collected;

[0051] S2.1.2, divide the training set, validation set and test set: the ratio of 70% training set, 10% validation set and 20% test set is still used;

[0052] S2.1.3, Feature selection: Feature selection is the process of selecting a subset of features that have the greatest impact on the model's predictive performance from the extracted clinical data features.

[0053] S2.1.4, AI-Gleason score prediction: The AI-Gleason score of all images can be obtained through the Gleason score prediction system, and this indicator will be included in the subsequent analysis together with the selected clinical features;

[0054] S2.1.5, Model selection: Select appropriate preprocessors and classification models based on the nature of the problem;

[0055] S2.1.6, Hyperparameter setting and model training: Use the training set data to train the model. During the training process, perform hyperparameter tuning as needed to optimize the performance of the model.

[0056] S2.1.7, Model Evaluation: Use the validation set to evaluate the performance of the model. The evaluation indicators of the classification task include AUC, sensitivity, specificity, accuracy, precision and F1 score indicators, and comprehensively select the optimal model.

[0057] S2.2 Reasoning phase, model application.

[0058] Preferably, in step S1.1.2,

[0059] Bias field correction uses N4 Bias Field Correction to eliminate intensity inhomogeneity in MR images;

[0060] The T2 sequence is used as the reference image, and the ADC and DWI sequences are used as floating images. The floating image is mapped to the reference image through the image registration algorithm Symmetric normalization to achieve spatial position registration of the three images.

[0061] Preferably, in step S1.1.4, the deep learning neural network model is a ResNet neural network model or a DenseNet neural network model; the neurons in the output layer use a linear activation function or a sigmoid function.

[0062] Preferably, in step S1.2.7, the generated visualization tools include prediction curves, scatter plots and Bland-Altman plots.

[0063] Preferably, the clinical data in step S2.1.1 include: patient age, height, weight, prostate volume, prostate-specific antigen density PSAD, free prostate-specific antigen fPSA, and total prostate-specific antigen tPSA.

[0064] Preferably, in step S2.1.5, the preprocessor includes: Box-Cox transformer, L1 norm regularization, L2 norm regularization, absolute maximum normalization, maximum minimum normalization, Quantile transformer, YeoJohnson transformer and Z score normalization; the classification model includes AdaBoost, Bagging decision tree, decision tree, Gaussian process, gradient descent tree, K nearest neighbor, logistic regression, partial least squares discriminant analysis, quadratic discriminant analysis, random forest, stochastic gradient descent, support vector machine, and XGBoost.

[0065] Preferably, the step S2.2 includes

[0066] S2.2.1, Data preparation: For new prostate cancer patients, collect their DWI, ADC and T2 image data, the prostate cancer lesion marking mask of the patient's T2 image, and clinical information;

[0067] S2.2.2, Feature extraction:

[0068] Image feature extraction: Use the same deep learning model Gleason score prediction system as in the training phase to extract the deep learning predicted Gleason score of the image;

[0069] Clinical feature selection: extract and select the same clinical features as those in the training phase;

[0070] S2.2.3, feature concatenation: concatenate the extracted image features and the selected clinical features into a feature vector;

[0071] S2.2.4, model reasoning: load the trained classifier model, input the concatenated feature vector into the model for prediction; the model outputs the prediction result, judging whether the patient has clinically significant prostate cancer or clinically insignificant prostate cancer;

[0072] S2.2.5, Evaluation and Interpretation: Use evaluation metrics to explain the model's prediction results on the test set, including AUC, sensitivity, specificity, accuracy, precision, and F1 score;

[0073] S2.2.6, Result Visualization: Generate visualization tools such as confusion matrix, ROC curve, calibration curve, decision curve, etc. based on the test results to better understand the performance of the model on different categories;

[0074] S2.2.7, Deploy model: Deploy the trained model to the actual application scenario for real-time prediction or batch processing.

[0075] The beneficial effects of the present invention are mainly embodied in:

[0076] (1) The application of deep learning technology in the prediction of Gleason score of prostate cancer has significantly improved the objectivity and consistency of the scoring process. The deep learning system can automatically process pathological images, avoid the influence of human factors on the scoring process, and improve the objectivity of the scoring. It can also reduce the variability between observers and provide more consistent and accurate scores.

[0077] (2) The deep learning system scores images through fixed algorithms and processes, ensuring the standardization and repeatability of the scoring process.

[0078] (3) The present invention innovatively combines regression loss with classification loss, combining the MSE loss function commonly used in regression tasks with the Focal loss function commonly used in classification tasks as the loss function for network training. Classification loss focuses on the correct prediction of the category, while regression loss focuses on the closeness between the predicted value and the true value. This combination helps the model understand the data from multiple perspectives, thereby improving its generalization ability.

[0079] (4) The present invention innovatively combines the Gleason score predicted by deep learning with a series of key clinical data information, such as age, prostate volume, prostate-specific antigen density (PSAD) and total prostate-specific antigen (tPSA), to jointly construct an efficient and accurate prostate cancer risk calculator. This calculator not only significantly improves the accuracy and reliability of predictions, but also effectively reduces the consumption of medical resources by reducing the steps of Gleason score collection. At the same time, this optimized process also shortens the waiting time for patients, bringing them a more convenient and efficient medical service experience.

[0080] (5) The present invention realizes automated risk prediction and diagnosis of prostate cancer in patients through conventional magnetic resonance imaging examinations combined with four conventional clinical characteristics, without the need for additional complex post-processing and equipment, greatly improving the overall convenience and reducing the cost of examinations. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 It is the overall workflow diagram of the present invention;

[0082] Figure 2 The algorithm model structure of the preprocessor and classifier is displayed. DETAILED DESCRIPTION

[0083] In order to better understand the present invention, the invention is further described below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the technical implementation of the present invention are shown in the accompanying drawings, rather than all structures.

[0084] The present invention discloses an intelligent prediction solution for clinically significant prostate cancer, which is used to realize the Gleason score of prostate cancer and provide guiding conclusions and suggestions for the prediction analysis of clinically significant prostate cancer (csPCA). After testing, the overall test results are good and the operation is interactive and convenient.

[0085] From the overall division of the workflow, the present invention is divided into two major system modules according to the sequence of the workflow: the Gleason score prediction system and the csPCA risk prediction system. The overall working mode is to first obtain a data set containing the Gleason score through the Gleason score prediction system, and then combine it with selected clinical data, such as age, prostate volume, prostate-specific antigen density (PSAD) and total prostate-specific antigen (tPSA) for analysis and processing, and finally give a predictive conclusion with guiding significance.

[0086] The following will introduce the specific working steps of the two major system modules in detail to demonstrate the core points of the implementation of the present invention.

[0087] 1. Gleason score prediction system

[0088] Its workflow includes two major stages: training stage and inference stage;

[0089] 1.1 Training Phase

[0090] 1.1.1, Data preparation: Collect a dataset containing prostate cancer patient images (DWI, ADC and T2) and the corresponding Gleason scores.

[0091] 1.1.2, Image preprocessing:

[0092] Bias field correction: Figure 1 As shown, bias field correction: All images use the public N4 bias field correction algorithm to eliminate the intensity inhomogeneity of MR images. Many image processing software and libraries provide the implementation of N4 bias field correction, such as SimpleITK, ANTs, etc. These software and libraries usually contain the source code or executable file of the N4 algorithm, and users can use the N4 bias field correction function by calling the corresponding function or command.

[0093] Registration: In order to realize the analysis based on multi-sequence information, we use the T2 sequence as the reference image, ADC and DWI sequences as floating images, and use the public ANTs registration algorithm. The parameters are selected for symmetric normalization, including affine and deformable transformations, and the mutual information is used as the optimization metric to map the floating image to the reference image, so as to achieve the purpose of consistent spatial position of the three sequence images of the same patient, namely T2, ADC, and DWI.

[0094] 1.1.3, Divide the training set, validation set and test set: use a ratio of 70% training set, 10% validation set and 20% test set.

[0095] 1.1.4, Model selection: Select a deep learning model suitable for regression tasks, such as ResNet, DenseNet, etc., but the output layer needs to be modified. The output layer should have only one neuron, and usually uses a linear activation function (such as the identity function) or a sigmoid function (if the predicted value is limited to a certain range). Adjust the model structure and hyperparameters according to task requirements. Hyperparameters include resolution, cropping size, maximum number of iterations, initial learning rate, decay rate, optimizer, batch size, etc. Of course, on this basis, hyperparameter data items and specific definition standards can be added according to actual needs.

[0096] Table 1 Example of hyperparameter settings for the Gleason score prediction system

[0097] Hyperparameters Gleason score prediction system Input Channels Four channels: T2+ADC+DWI+prostate cancer lesion marking mask Resolution [0.39, 0.39, 3.6] Cutting size [160, 112, 16] Maximum number of iterations 6000 Initial learning rate 1e-4 Decay rate 0.1 Optimizer Adaptive Moment Estimator Optimizer (Adam) Batch Number 16

[0098] 1.1.5, Loss function selection: Choose an appropriate loss function L loss to measure the difference between the model output and the true label. Since the Gleason score is generally between 2-10, each score represents a different degree of malignancy of prostate cancer tissue. Therefore, technically speaking, Gleason score prediction can be regarded as both a multi-classification task and a regression task. This invention innovatively combines regression loss with classification loss, and converts the MSE loss function L commonly used in regression tasks into MSE The Focal loss function L commonly used in classification tasks Focal Combined as the loss function for network training. Combining classification loss and regression loss can provide richer information for model training. Classification loss focuses on the correct prediction of the category, while regression loss focuses on the closeness between the predicted value and the true value. This combination helps the model understand the data from multiple perspectives, thereby improving its generalization ability. The specific definitions are as follows:

[0099] L loss = ωL MSE + (1-ω) L focal (1)

[0100] L loss Represents the loss function of network training, L MSE represents the mean square error loss, L focal Represents focal loss. ω is the weight balance parameter between mean square error loss and focal loss, and its value range is 0-1. When ω is 0, the loss function of network training is focal loss; when ω is 1, the loss function of network training is mean square error loss; when ω is other values ​​between 0-1, the loss function of network training is the weighted sum of mean square error loss and focal loss. According to experience, ω is generally taken as 0.5.

[0101] 1.1.6, Data reading: The number of samples read each time is determined by the batch size of the hyperparameter, which refers to the number of samples processed simultaneously during each iteration of training. For example, if the batch size in Table 1 is 16, then the number of samples processed simultaneously is 16. Randomly select a batch size of samples from the training data set, and read the images of these samples after the second step of preprocessing into the model.

[0102] 1.1.7 Cropping and resampling: Based on the prostate cancer annotation results, the position of the prostate cancer in the entire image can be obtained, including the center point coordinates and length, width and height information. Within the range of the prostate cancer, image blocks of fixed size and resolution are randomly sampled. The resolution and size are determined by the hyperparameters resolution and cropping size, as shown in Table 1.

[0103] 1.1.8, Data preprocessing: Perform data enhancement operations such as rotation, flipping, translation, etc., or other enhancement operations. Then, using the patient image as a unit, use adaptive normalization processing to remove voxels with MR image intensity values ​​higher than 99% and lower than 1%, and use maximum and minimum normalization processing to unify all image intensity values ​​between 0 and 1.

[0104] 1.1.9, forward propagation: A batch_size of preprocessed images are input into the model. Through the forward propagation operation, the image will pass through each neuron in the network layer by layer to predict the Gleason score.

[0105] 1.1.10, Loss calculation: Calculate the loss between the network predicted Gleason score and the true Gleason score according to the selected loss function and calculate the gradient, and use the back propagation algorithm to update the model parameters.

[0106] 1.1.11, deep learning network model training: repeat steps 7 to 11, and traverse the entire training set data once to complete one iteration of training. Repeat this step continuously until the maximum number of iterations is reached, and the entire model training process is completed. The present invention optimizes the model parameters through loss calculation and back propagation algorithm, so that the model can better predict the Gleason score of the input sample. During the training process, the model performance is evaluated on the validation set generally every 20 iterations.

[0107] 1.1.12, Model evaluation and tuning: Evaluate model performance on the validation set, generally calculate parameters such as mean absolute error, mean square error, R square coefficient, Pearson coefficient, and comprehensively select the optimal model.

[0108] 1.2 Reasoning Phase

[0109] 1.2.1, Image preprocessing: Same as the training stage, including bias field correction and registration.

[0110] Bias field correction: All images use the public N4 bias field correction algorithm to eliminate the intensity inhomogeneity of MR images. Many image processing software and libraries provide the implementation of N4 bias field correction, such as SimpleITK, ANTs, etc. These software and libraries usually contain the source code or executable files of the N4 algorithm, and users can use the N4 bias field correction function by calling the corresponding function or command. The formula is as follows:

[0111]

[0112] in is the observed MRI image, is the real signal, is a low-frequency spatially varying bias field (usually caused by magnetic field inhomogeneities), is additive noise. Estimation by iterative optimization And correct the image.

[0113] Registration: In order to realize the analysis based on multi-sequence information, we use the T2 sequence as the reference image, ADC and DWI sequences as floating images, and use the public ANTs registration algorithm. The parameters are selected for symmetric normalization, including affine and deformable transformations, and the mutual information is used as the optimization metric to map the floating image to the reference image, so as to achieve the purpose of consistent spatial position of the three sequence images of the same patient, namely T2, ADC, and DWI.

[0114]

[0115] in is the reference image, is a floating image, T is an affine transformation matrix, and NMI is the normalized mutual information. The goal of registration is to find the optimal affine transformation , so that NMI is minimized.

[0116] 1.2.2, Data reading and positioning: read the prostate cancer lesion marking mask of the test sample's T2, ADC, DWI three-series images and T2 image into the model. Calculate the bounding box parameters of the prostate cancer lesion in the entire image based on the prostate cancer lesion marking mask, and then calculate the center point coordinates and length, width and height information according to the following formula.

[0117]

[0118]

[0119] are the minimum and maximum coordinates on the x, y, and z axes respectively.

[0120] 1.2.3, Cropping and resampling: With the coordinates of the center point of the prostate cancer lesion marking mask in the original image as the center, sample image blocks of fixed size and resolution. The resolution and size are determined by the hyperparameters resolution and cropping size during training.

[0121] Specifically, it can be: taking the coordinate c of the center point of the prostate cancer lesion marking mask in the original image as the center, sampling an image block with a fixed size of [160, 112, 16] and a fixed resolution of [0.39, 0.39, 3.6] according to the training hyperparameters.

[0122] , where I is the image before cropping, including the three-series image of the test sample, T2 image, ADC image, DWI image and the prostate cancer lesion marking mask registered with the T2 sequence, and c is the coordinate of the center point of the prostate cancer lesion marking mask in the original image.

[0123] Resampling is done using trilinear interpolation:

[0124]

[0125] in Indicates the value of the new position after trilinear interpolation. For each point involved in the interpolation The interpolation weights. Representing coordinates The original value at .

[0126] 1.2.4, Data preprocessing: Preprocess the read image blocks according to the image normalization method set during the training process;

[0127] Specifically, the read image blocks are normalized at the channel level, that is, each modality is independently adaptively normalized. The following operations are performed on the three-series image input blocks of the test sample, T2, ADC, and DWI: the 1% quantile and 99% quantile of the input image block are calculated as the minimum value min and maximum value max of the normalization, and then the mean mean and variance stddev are calculated according to the following formula, and finally z-score normalization is performed.

[0128]

[0129]

[0130]

[0131] 1.2.5, forward propagation: load the trained deep learning regression model, input the preprocessed image blocks into the model for forward propagation, and calculate the Gleason score prediction value;

[0132] The output layer of the deep learning regression network is a linear regression layer, and the formula is as follows:

[0133]

[0134] in, It is a deep learning network feature extractor. and b are the weight and intercept of the linear regression layer respectively.

[0135] Since the Gleason score is an integer, the integer closest to the predicted value is usually selected as the final Gleason score prediction result.

[0136]

[0137] As in the case example: Model output , after rounding, the final forecast , the true label .

[0138] 1.2.6, Evaluation index calculation: Calculate the performance index of the model on the test set based on the difference between the predicted results and the true labels, such as mean absolute error MAE, mean square error MSE, R square coefficient, Pearson coefficient r. These indicators can help evaluate the performance of the model on regression tasks.

[0139]

[0140]

[0141]

[0142]

[0143] Where N is the number of samples, : The true value of the i-th sample, : The predicted value of the i-th sample; : The average value of the true value; : The mean of the predicted values.

[0144] 1.2.7, Result Visualization: Generate visualization tools such as prediction curves, scatter plots, and Bland-Altman plots based on the test results to better understand the performance of the model on the regression task.

[0145] 2. csPCA risk prediction system

[0146] It also includes training phase and inference phase

[0147] 2.1 Training Phase

[0148] 2.1.1 Data preparation: The above-mentioned datasets containing prostate cancer patient images (DWI, ADC, and T2) and the corresponding Gleason scores have been collected. In addition, the patient's age, height, weight, prostate volume, prostate-specific antigen density (PSAD), free prostate-specific antigen (fPSA), total prostate-specific antigen (tPSA), and other possible clinical information need to be collected.

[0149] 2.1.2, Divide the training set, validation set and test set: The ratio of 70% training set, 10% validation set and 20% test set is still used. It is consistent with the data grouping of the above tasks.

[0150] 2.1.3, Feature selection: Feature selection is the process of selecting the feature subset that has the greatest impact on the model prediction performance from the extracted features. The purpose of feature selection is to reduce the number of features, improve the efficiency of model training and avoid overfitting. In the present invention, univariate analysis and multivariate analysis are used to select clinical features. Finally, the more important features selected include age, prostate volume, PSAD, and tPSA.

[0151] 2.1.4, AI-Gleason score prediction (Gleason score predicted by deep learning): The AI-Gleason score of all images can be obtained through the Gleason score prediction system, and this indicator will be included in subsequent studies together with the selected clinical features.

[0152] 2.1.5, Model selection: According to the nature of the problem (distinguishing clinically significant prostate cancer csPCA and clinically ineffective prostate cancer cisPCA), select the appropriate preprocessor and classification model. Preprocessing methods such as Figure 2 As shown in , including Box-Cox transformer, L1 norm regularization, L2 norm regularization, absolute maximum normalization, maximum minimum normalization, Quantile transformer, YeoJohnson transformer and Z score normalization, preprocessors can be used one by one or in combination. Classification models include AdaBoost, Bagging decision tree, decision tree, Gaussian process, gradient descent tree, K nearest neighbor, logistic regression, partial least squares discriminant analysis, quadratic discriminant analysis, random forest, stochastic gradient descent, support vector machine, and XGBoost.

[0153] 2.1.6, Hyperparameter setting and model training: Use training set data to train the model. During the training process, hyperparameter tuning may be required. For example, AdaBoost, Bagging decision tree, decision tree, gradient descent tree, random forest, XGBoost, etc. need to adjust the maximum depth and the number of weak classifiers, Gaussian process needs to adjust the number of refitting times, K nearest neighbor needs to adjust the number of neighbors, logistic regression needs to adjust the penalty factor, penalty parameter, tolerance, etc., stochastic gradient descent needs to adjust the learning rate and loss function, and support vector machine needs to adjust the kernel function to optimize the performance of the model.

[0154] 2.1.7, Evaluation model: Use the validation set to evaluate the performance of the model. The evaluation indicators of the classification task include AUC, sensitivity, specificity, accuracy, precision and F1 score indicators, and comprehensively select the optimal model.

[0155] 2.2. Reasoning phase, model application

[0156] S2.2.1, Data preparation: For new prostate cancer patients, collect their DWI, ADC and T2 image data, the prostate cancer lesion marking mask of the patient's T2 image, and clinical information (such as age, height, weight, prostate volume, PSAD, fPSA, tPSA, etc.).

[0157] S2.2.2, Feature extraction:

[0158] Image feature extraction: The deep learning predicted Gleason score of the image is extracted using the same deep learning model (Gleason score prediction system) as in the training phase.

[0159] Clinical feature selection: The same clinical features as those in the training phase (e.g., age, prostate volume, PSAD, tPSA) were extracted and selected.

[0160] S2.2.3, Feature concatenation: Concatenate the extracted image features (Gleason score predicted by deep learning) and the selected clinical features into a feature vector.

[0161] S2.2.4, Model Inference: Load the trained classifier model and input the concatenated feature vector into the model for prediction. The model outputs the prediction result, such as whether the patient belongs to csPCA (clinically significant prostate cancer) or cisPCA (clinically insignificant prostate cancer).

[0162] S2.2.5, Evaluation and Interpretation: Use evaluation indicators to explain the prediction results of the model on the test set, including AUC, sensitivity, specificity, accuracy, precision and F1 score. The calculation formula of the indicator parameters is as follows:

[0163] (1) AUC

[0164]

[0165] N: number of negative samples

[0166] P: number of positive samples

[0167] : Prediction probability of negative samples

[0168] : Prediction probability of positive sample

[0169] II: indicator function, 1 if the condition is true, 0 otherwise

[0170] (2) Sensitivity

[0171]

[0172] TP: True positive, the actual sample is positive and the prediction is also positive.

[0173] FN: False negative, which is actually a positive sample but predicted to be a negative sample.

[0174] (3) Specificity

[0175]

[0176] TN: True negative, which is actually a negative sample and is predicted to be a negative sample.

[0177] FP: False positive, which is actually a negative sample but predicted to be a positive sample.

[0178] (4) Accuracy

[0179]

[0180] (5) Accuracy

[0181]

[0182] (6) F1 score

[0183]

[0184] S2.2.6, Result Visualization: Generate visualization tools such as confusion matrix, ROC curve, calibration curve, decision curve, etc. based on the test results to better understand the performance of the model on different categories.

[0185] S2.2.7, Deploy model: Deploy the trained model to the actual application scenario for real-time prediction or batch processing.

[0186] Analysis of specific case application examples

[0187] The following will combine specific cases to demonstrate the model's reasoning process and output results.

[0188] 1. Data preparation

[0189] Suppose we have a new prostate cancer patient, collect his DWI, ADC and T2 image data and the prostate cancer lesion marking mask registered by T2 image, and his clinical information is as follows:

[0190] Age: 65

[0191] Height: 175 cm

[0192] Weight: 80 kg

[0193] Prostate volume: 30 cm³

[0194] PSAD: 0.15

[0195] fPSA: 0.5 ng / mL

[0196] tPSA: 6 ng / mL

[0197] 2. Image Preprocessing

[0198] The three sequence images were corrected using the N4 bias field correction algorithm.

[0199] The T2 sequence was used as the reference image, and the ADC and DWI sequences were used as floating images. ANTs were used for registration to make the spatial positions of the three sequence images consistent.

[0200] 3. Feature extraction

[0201] Image feature extraction: Using the Gleason score prediction system, image reading and positioning, cropping and resampling, and image normalization were performed in sequence, and the patient's Gleason score was obtained as 4.

[0202] Selection of clinical characteristics: Age, prostate volume, PSAD, and tPSA were selected as clinical characteristics.

[0203] 4. Feature stitching

[0204] Feature vector: Clinical features and Gleason score are concatenated into a feature vector x=[65,30,0.15,6,4].

[0205] 5. Model Reasoning

[0206] Model loading: Load a trained classifier model, such as a random forest classifier.

[0207] Feature input: Input the feature vector x into the model.

[0208] Output prediction results: The model outputs the prediction results.

[0209] Assuming that the probability value output by the model is [p(csPCA), p(cisPCA)] = [0.85, 0.15], it can be concluded that the probability that the patient is predicted to have csPCA is 85%, which is clinically significant prostate cancer.

Claims

1. A medical intelligent prediction system for prostate cancer based on deep learning, characterized by: It includes a Gleason score prediction system and a csPCA risk prediction system; the csPCA risk prediction system can combine clinical data and the data set obtained by the Gleason score prediction system to perform comprehensive prediction evaluation; The working steps of the Gleason score prediction system are as follows: S1.

1. Training phase S1.1.1, Data preparation: Collect a dataset containing images of prostate cancer patients and their corresponding Gleason scores; images of prostate cancer patients include DWI images, ADC images, and T2 images; S1.1.2, Image preprocessing: Perform bias field correction and registration on images; S1.1.3, divide the training set, validation set and test set into the following proportions: 70% training set, 10% validation set and 20% test set; S1.1.4, Model selection and hyperparameter setting: Select a deep learning neural network model suitable for regression tasks; Adjust the model structure and hyperparameters according to the task requirements, and modify the output layer to ensure that the output layer should have only one neuron; the hyperparameters include at least: resolution, cropping size, maximum number of iterations, initial learning rate, decay rate, optimizer, and batch size; S1.1.5, loss function selection: Combined with the MSE loss function L commonly used in regression tasks MSE The Focal loss function L commonly used in classification tasks Focal According to formula (1), select an appropriate loss function L loss To measure the difference between the model output and the true label; L loss = ωL MSE + (1-ω) L focal (1) In formula (1), L loss Represents the loss function of network training, L MSE represents the mean square error loss, L focal Represents focal loss; ω is the weight balance parameter between mean square error loss and focal loss, and its value range is 0-1; when ω is 0, the loss function of network training is focal loss; when ω is 1, the loss function of network training is mean square error loss; when ω is other values ​​between 0-1, the loss function of network training is the weighted sum of mean square error loss and focal loss; S1.1.6, data reading: The number of samples read each time is determined by the batch size of the hyperparameter, which is the number of samples processed simultaneously during each iteration of training; a batch size of samples is randomly selected from the training data set, and the images of these samples preprocessed by S1.1.2 are read into the model; S1.1.7, Cropping and resampling: According to the prostate cancer annotation results, the position of the prostate cancer in the entire image is obtained. Within the range of the prostate cancer, image blocks of fixed size and resolution are randomly sampled. The resolution and sampling size are determined by the hyperparameters: resolution and cropping size; S1.1.8, data preprocessing: perform data enhancement operations on the data; then use the patient image as a unit, use adaptive normalization processing to remove voxels with MR image intensity values ​​higher than 99% and lower than 1%, and use maximum and minimum normalization processing to unify all image intensity values ​​between 0 and 1; S1.1.9, forward propagation: a batch of preprocessed images are input into the model. Through the forward propagation operation, the images pass through each neuron in the network layer by layer to predict the Gleason score; S1.1.10, Loss calculation: Calculate the loss between the network predicted Gleason score and the true Gleason score according to the selected loss function and calculate the gradient, and use the back propagation algorithm to update the model parameters; S1.1.11, deep learning network model training: repeat S1.1.6 to S1.1.10, traverse the entire training set data once to complete one iteration of training; then repeat this step until the maximum number of iterations is reached, completing the entire model training process; S1.1.12, Model evaluation and tuning: Evaluate model performance on the validation set, obtain mean absolute error, mean square error, R-square coefficient, Pearson coefficient, and select the optimal model; S1.

2. Reasoning stage S1.2.1, Image preprocessing: The method is the same as S1.1.2, including bias field correction and registration; S1.2.2, data reading: read the three image sequences of the test sample: DWI image, ADC image and T2 image, as well as the prostate cancer lesion marking mask of the T2 image into the model; according to the prostate cancer lesion marking mask, its position in the whole image can be obtained, including the center point coordinates and length, width and height information; S1.2.3, Cropping and resampling: Taking the coordinates of the center point of the prostate cancer lesion marking mask in the original image as the center, sample image blocks of fixed size and resolution. The resolution and size are determined by the hyperparameters resolution and cropping size during training; S1.2.4, data preprocessing: preprocess the read image blocks according to the image normalization method set in the training phase; S1.2.5, forward propagation: load the trained deep learning network model, input the preprocessed image blocks into the model for forward propagation, and calculate the Gleason score prediction value; select the integer closest to the prediction value as the final Gleason score prediction result; S1.2.6, evaluation index calculation: Calculate the performance index of the model on the test set based on the difference between the predicted results and the true labels, and obtain the mean absolute error, mean square error, R square coefficient, and Pearson coefficient; S1.2.7, Result Visualization: Generate visualization tools based on test results to better understand the performance of the model on the regression task; The working steps of the csPCA risk prediction system are as follows: S2.1 Training Phase S2.1.1, data preparation: In addition to the data set containing three-sequence DWI images, ADC images and T2 images of prostate cancer patients and the corresponding Gleason scores collected in step S1 above, clinical data needs to be collected; S2.1.2, divide the training set, validation set and test set: the ratio of 70% training set, 10% validation set and 20% test set is still used; S2.1.3, Feature selection: Feature selection is the process of selecting a subset of features that have the greatest impact on the model's predictive performance from the extracted clinical data features; S2.1.4, AI-Gleason score prediction: The AI-Gleason score of all images can be obtained through the Gleason score prediction system, and this indicator will be included in the subsequent analysis together with the selected clinical features; S2.1.5, Model selection: Select appropriate preprocessors and classification models based on the nature of the problem; S2.1.6, Hyperparameter setting and model training: Use the training set data to train the model. During the training process, perform hyperparameter tuning as needed to optimize the performance of the model; S2.1.7, Evaluate the model: Use the validation set to evaluate the performance of the model. The evaluation indicators of the classification task include AUC, sensitivity, specificity, accuracy, precision and F1 score indicators, and comprehensively select the optimal model; S2.2 Reasoning phase, model application.

2. The deep learning-based medical intelligent prediction system for prostate cancer according to claim 1, characterized in that: In step S1.1.2, Bias field correction uses N4 Bias Field Correction to eliminate intensity inhomogeneity in MR images; The T2 sequence is used as the reference image, and the ADC and DWI sequences are used as floating images. The floating image is mapped to the reference image through the image registration algorithm Symmetric normalization to achieve spatial position registration of the three images.

3. The deep learning-based medical intelligent prediction system for prostate cancer according to claim 1, characterized in that: In step S1.1.4, the deep learning neural network model is a ResNet neural network model or a DenseNet neural network model; the neurons of the output layer use a linear activation function or a sigmoid function.

4. The deep learning-based medical intelligent prediction system for prostate cancer according to claim 1, characterized in that: In step S1.2.7, the generated visualization tools include prediction curves, scatter plots, and Bland-Altman plots.

5. The deep learning-based medical intelligent prediction system for prostate cancer according to claim 4 is characterized in that: The clinical data in step S2.1.1 include: patient age, height, weight, prostate volume, prostate-specific antigen density PSAD, free prostate-specific antigen fPSA, and total prostate-specific antigen tPSA.

6. The deep learning-based medical intelligent prediction system for prostate cancer according to claim 5, characterized in that: In the step S2.1.5, the preprocessor includes: Box-Cox transformer, L1 norm regularization, L2 norm regularization, absolute maximum normalization, maximum minimum normalization, Quantile transformer, YeoJohnson transformer and Z score normalization; the classification model includes AdaBoost, Bagging decision tree, decision tree, Gaussian process, gradient descent tree, K nearest neighbor, logistic regression, partial least squares discriminant analysis, quadratic discriminant analysis, random forest, stochastic gradient descent, support vector machine, and XGBoost.

7. The deep learning-based medical intelligent prediction system for prostate cancer according to claim 6, characterized in that: The step S2.2 includes S2.2.1, Data preparation: For new prostate cancer patients, collect their DWI, ADC and T2 image data, as well as the prostate cancer lesion marking mask of the patient's T2 image, and clinical information; S2.2.2, Feature extraction: Image feature extraction: The Gleason score prediction system using the same deep learning model as in the S1.1 training phase was used to extract the deep learning predicted Gleason score of the image; Clinical feature selection: Extract and select the same clinical features as in the S2.1 training phase; S2.2.3, feature concatenation: concatenate the extracted image features and the selected clinical features into a feature vector; S2.2.4, model reasoning: load the trained classifier model, input the concatenated feature vector into the model for prediction; the model outputs the prediction result, judging whether the patient has clinically significant prostate cancer or clinically insignificant prostate cancer; S2.2.5, Evaluation and Interpretation: Use evaluation metrics to explain the prediction results of the model on the test set. Evaluation metric parameters include AUC, sensitivity, specificity, accuracy, precision, and F1 score; S2.2.6, Result Visualization: Generate visualization tools based on test results to better understand the performance of the model on different categories; S2.2.7, Deploy model: Deploy the trained model to the actual application scenario for real-time prediction or batch processing.

Images