Kidney tumor subtype distinguishing method based on 3D deep learning

Abstract

The invention discloses a kidney tumor subtype distinguishing method based on 3D deep learning. The kidney tumor subtype distinguishing method comprises the following steps of S1, obtaining patient data; s2, image processing and segmentation: downloading CT images of all patient data, carrying out format conversion, and then carrying out image segmentation and processing; step S3, image omics feature extraction; s4, depth feature extraction is carried out to obtain depth features, and then model construction is carried out based on radiomics features and the depth features; s5, obtaining an optimal distinguishing model, and predicting the type of the kidney tumor subtype through the optimal distinguishing model, the structure is novel, the conception is ingenious, the operation is simple and convenient, the depth feature can be well extracted, the benign and malignant kidney tumor subtypes can be well distinguished, and the method is suitable for popularization and application. The function of combining the prediction probability with clinical features, genetic markers and histopathology is added, and the method is used for carrying out auxiliary decision making on cases with medium confidence coefficient prediction in clinical practice.

Description

Technical Field

[0001] The present invention belongs to the field of medical detection technology and relates to a method for distinguishing kidney tumor subtypes based on 3D deep learning. Background Art

[0002] In 2022, kidney cancer caused 434,840 new cases and 155,953 deaths worldwide. Renal cell carcinoma (RCC) is the most common solid tumor in the kidney, accounting for about 90% of all renal malignancies, of which clear cell carcinoma (ccRCC) is the most common, accounting for 70%, followed by papillary carcinoma (pRCC) accounting for 10%-15%, and chromophobe cell carcinoma (chRCC) accounting for 5%. In the past two decades, the incidence of RCC has increased by about 2% per year, which may be related to the prevalence of known risk factors and the widespread use of imaging such as ultrasound and CT.

[0003] The decision on whether to resect a cystic or solid renal lesion often relies on clinical imaging studies in the absence of a definitive histologic diagnosis. Studies have shown that approximately 20% of surgically resected renal lesions are benign, including renal oncocytomas, angiomyolipomas (AMLs), complex cysts, and other benign lesions. The percentage of benign lesions increases as the diameter of the lesion decreases. This poses a significant challenge in determining appropriate treatment options for all suspicious lesions, especially small renal lesions, defined as ≤4.0 cm in diameter. Improving the diagnosis and differentiation of small renal lesions has been identified as a key research priority for RCC by international research initiatives. In addition, accurate classification of renal tumor subtypes in images is critical, as treatment planning and prognosis prediction depend largely on the pathologic subtype of RCC.

[0004] Traditionally, enhanced CT is the routine method used in clinical practice to characterize renal lesions based on tumor enhancement features. However, there is significant overlap in image features among renal tumor subtypes, making subtype classification challenging and leading to inter-observer variability. In recent years, deep learning has made significant progress in image analysis, revolutionizing the medical image processing workflow. In the field of renal lesions, deep learning techniques have been used for tumor segmentation and classification. However, previous diagnostic models mainly relied on two-dimensional images and classified lesions as benign or malignant, or classified them into the three main RCC subtypes (ccRCC, pRCC, and chRCC). In addition, most studies did not perform subgroup analysis of small renal lesions.

[0005] Therefore, it is necessary to comprehensively consider the impact of CT imaging phase, ROI segmentation strategy and background information on diagnostic accuracy and obtain a renal tumor subtype differentiation method based on 3D deep learning by using conventional CT images to help medical staff distinguish renal tumor subtypes (benign, ccRCC, pRCC and chRCC). Summary of the invention

[0006] In response to the above problems, the present invention proposes a method for distinguishing renal tumor subtypes based on 3D deep learning, which effectively solves the problems in the prior art.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A method for distinguishing renal tumor subtypes based on 3D deep learning, comprising the following steps:

[0009] Step S1, obtaining patient data, and screening CT images of suitable patients with renal disease according to inclusion criteria;

[0010] Step S2, image processing and segmentation, downloading CT images of all patient data and converting their formats, and then performing image segmentation and processing;

[0011] Step S3, radiomics feature extraction, after normalizing and discretizing all patient data, radiomics feature extraction is performed;

[0012] Step S4: deep feature extraction to obtain deep features, and then construct a model based on the radiomics features and the deep features;

[0013] Step S5: Obtain an optimal discrimination model, and use the optimal discrimination model to predict the type of renal tumor subtype.

[0014] Preferably, in step S1, the inclusion criteria include the following:

[0015] Standard S101: renal lesions confirmed by histopathology to be benign, clear cell renal carcinoma, papillary renal carcinoma, or chromophobe cell carcinoma;

[0016] Standard S102: Provide preoperative CT axial images that include at least the arterial and venous phases;

[0017] Standard S103, CT image quality is suitable for analysis, without severe motion or artifacts.

[0018] Preferably, in step S2, the CT images of all patient data are downloaded in DICOM format with their original size and resolution, and saved in NiFTI format after anonymization.

[0019] Preferably, in step S2, the image segmentation and processing includes the following steps:

[0020] Step S201, manually outlining the arterial phase image layer by layer to obtain voxel-level segmentation labels;

[0021] Step S202: The voxel-level segmentation labels obtained in step S201 are manually reviewed and improved to obtain the tumor region.

[0022] Preferably, in step S4, deep feature extraction includes the following steps:

[0023] Step S401, pre-training, cropping a cube containing both the ROI and background information according to the range of the ROI, filling the non-foreground area in the cropped data with zeros, and using a predefined vector to translate the lesion center before re-cropping to enhance the training data, thereby limiting overfitting;

[0024] Step S402, fine-tuning, using the data of the training group for model parameter optimization, and minimizing the loss function through the Adam optimizer to identify benign and malignant renal tumor subtypes;

[0025] Step S403, deep feature extraction, first, adjust the window width and window position, then use linear transformation to normalize the pixel values ​​of the image, and finally, extract the deep features from the fully connected layer before the SoftMax layer through forward propagation calculation.

[0026] Preferably, in step S5, obtaining the optimal differentiation model comprises the following steps:

[0027] Step S501, high confidence and medium confidence prediction, using the output probability of the model as a confidence measure, and dividing the prediction into high confidence and medium confidence according to the specified threshold t, for medium confidence prediction, retaining the first two predictions of the model, calculating indicators including the proportion of high confidence predictions, the accuracy of high confidence predictions and the accuracy of medium confidence predictions, determining the optimal threshold t, and selecting the average value with the highest threshold t;

[0028] Step S502, model evaluation and statistical analysis, continuous variables are expressed as medians, categorical variables are expressed as frequencies, continuous data are analyzed using Student's t test, categorical data are analyzed using chi-square test, balanced accuracy, accuracy and area under the receiver operating characteristic curve are used to evaluate the diagnostic accuracy of the model, the DeLong test is used to determine the 95% confidence interval of the area under the curve and compare the differences between different areas under the curve to obtain the optimal discrimination model.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention can accelerate the training convergence speed through pre-training, fine-tuning and feature extraction in deep feature extraction, and obtain better training results through fine-tuning.

[0031] 2. The present invention showed good separation ability of renal tumor subtypes in the entire cohort and small renal lesions through a multi-phase classifier.

[0032] 3. The present invention enhances the clinical informativeness of the model output through high-confidence and medium-confidence predictions. High-confidence predictions show high performance in the entire cohort and small renal lesions, and can provide reliable predictions. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the workflow of the present invention.

[0034] Figure 2 This is a schematic diagram of the present invention using balanced accuracy as an indicator for classifier evaluation.

[0035] Figure 3 Schematic diagram of the effect of the multi-phase classifier in the present invention in the verification cohort.

[0036] Figure 4 This is the patient data in the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] The following is combined with Figure 1-Figure 4 The specific implementation modes of the present invention are described in further detail.

[0039] The present invention comprises the following steps:

[0040] Step S1, obtaining patient data, screening CT images of suitable patients with renal lesions according to the inclusion criteria, and dividing all patient data into a model building group and a test group, wherein the model building group is divided into a training group and a validation group;

[0041] In step S1, the inclusion criteria include the following:

[0042] Standard S101: renal lesions confirmed by histopathology to be benign, clear cell renal carcinoma, papillary renal carcinoma, or chromophobe cell carcinoma;

[0043] Standard S102: Provide preoperative CT axial images that include at least the arterial and venous phases;

[0044] Standard S103, CT image quality suitable for analysis, no severe motion or artifacts;

[0045] In step S1, the ratio of the number of images in the training group to that in the verification group is 7:3;

[0046] In this embodiment, from 2011 to 2023. A total of 2780 patients who underwent preoperative enhanced CT examination, underwent partial nephrectomy in three medical centers and had renal tumors confirmed by histopathology after surgery were collected, of which 1419 lesions in Zhengzhou Hospital were divided into a training group (987 cases, 70%) and a validation group (432 cases, 30%). The data of Beijing Hospital (348 renal lesions) and Xi'an Hospital (312 renal lesions) constituted the test group.

[0047] Step S2, image processing and segmentation, downloading CT images of all patient data and converting their formats, and then performing image segmentation and processing;

[0048] In step S2, the CT images of all patient data will be downloaded in DICOM format with their original size and resolution. The CT images will be loaded into ITK-SNAP (version 3.8.0), anonymized and saved in NiFTI format. Since the arterial phase can provide the clearest tumor contour, it is selected as the reference target for subsequent registration with other phases;

[0049] In step S2, the image segmentation and processing includes the following steps:

[0050] Step S201: Under the supervision of a radiologist with at least 10 years of experience in abdominal radiology, three annotators outline the arterial phase images layer by layer to obtain voxel-level segmentation labels;

[0051] Step S202, the voxel-level segmentation labels obtained in step S201 are submitted to two other radiologists for review and improvement to obtain the intra-tumor region (ITR);

[0052] In this example, after generating the intra-tumor region (ITR), the arterial phase image was resampled to a voxel size of 1×1×1 mm using the sitkBSpline interpolator. Subsequently, the ITR was created by eroding the original ITR using the scipy.ndimage module (version: 1.11.3). -3mm The ITR is obtained by dilating the original ITR using the scipy.ndimage module (version: 1.11.3) +3mm and ITR +5mm .

[0053] The unenhanced and venous phase images were co-registered with the arterial phase images using the ITKElastix software package (version 0.17.0). The registration process involved a combination of translation (2000 iterations), rigid (1000 iterations), affine (1000 iterations), and B-spline transformations (1000 iterations) to ensure spatial consistency between the different phases.

[0054] Step S3, radiomics feature extraction, after normalizing and discretizing all patient data, radiomics feature extraction is performed;

[0055] In this embodiment, the z-score method is used for normalization, and the voxel intensity is discretized using 25 Hounsfield units. For the plain scan, arterial, and venous phase images, all four ROIs (ITR, ITR -3mm 、ITR +3mm and ITR +5mm ) for radiomics feature extraction.

[0056] Step S4: deep feature extraction to obtain deep features, and then construct a model based on the radiomics features and the deep features;

[0057] In step S4, deep feature extraction includes the following steps: pre-training, fine-tuning and feature extraction of 3D-ResNet34;

[0058] Step S401: pre-training. Before deep feature extraction, the ROI range (ITR, ITR -3mm 、ITR +3mm and ITR +5mm ) cropped cubes. These cropped cubes contain both ROIs and background information. At the same time, non-foreground regions in the cropped data are filled with zeros. We test the impact of background information on diagnostic accuracy.

[0059] To limit overfitting, the lesion center was shifted and re-cropped using a predefined vector (x,y,z): (3,3,3), (3,3,-3), (3,-3,3), (-3,3,3), (3,-3,-3), (-3,-3,3), (-3,3,-3), and (-3,-3,-3) mm. Image-based augmentation, including 3D random rotation and elastic transformation, was also applied, while certain augmentations were retained for images of benign lesions and ccRCC cases to ensure class balance.

[0060] The 3D-ResNet34 model is pre-trained on 23 medical datasets, which can accelerate the training convergence speed and achieve better results in the subsequent fine-tuning process than networks trained on natural videos or trained from scratch;

[0061] Step S402, fine-tuning, using the data of the training group for model parameter optimization, and minimizing the loss function through the Adam optimizer to identify benign and malignant renal tumor subtypes;

[0062] In this example, the output dimension of the SoftMax layer was set to 4, and the Adam optimizer was used with a batch size of 32. In the first 50 epochs, all model parameters except the fully connected layers were fixed, with a learning rate of 1e-4 and a weight decay of 0.0002. In the next 200 epochs, we fine-tuned the parameters of the entire model based on the back-propagated error, with a learning rate of 1e-5 and a weight decay of 0.0002 to identify benign and malignant renal tumor subtypes.

[0063] Step S403, feature extraction, first, adjust the window width and window level, then use linear transformation to normalize the pixel values ​​of the image, and finally, extract the depth feature from the fully connected layer before the SoftMax layer through forward propagation calculation;

[0064] In this embodiment, first, the window level and window width are set to 30 and 260 Hounsfield units, respectively. Then the pixel values ​​of the image are normalized to the range [0, 1] using a linear transformation. Finally, the deep features that can be calculated by forward propagation are extracted from the fully connected layer before the SoftMax layer. Each ROI (ITR, ITR -3mm 、ITR +3mm and ITR +5mm ) can extract 512 deep features;

[0065] Step S5, obtaining an optimal discrimination model, and predicting the type of renal tumor subtype by using the optimal discrimination model;

[0066] In this example, a stacked ensemble model (AutoGluon-Tabular classifier, version 0.8.2) was used to construct radiomics features and deep feature prediction models, respectively. Hyperparameters were optimized using the validation cohort. AutoGluon was used with the parameters set to "eval_metric = 'balanced_accuracy', and hyperparameters = 'NN_TORCH', while all other parameters were kept at default settings. Feature importance scores were generated in the validation dataset using the "feature_importance" function (default parameters) of the AutoGluon-Tabular classifier. A list of features with positive feature importance scores and P < 0.05 was used to predict benign and malignant renal tumor subtypes;

[0067] In step S5, model verification includes the following steps:

[0068] Step S501, high confidence and medium confidence prediction, using the output probability of the model as a confidence measure, and dividing the predictions into high confidence and medium confidence according to the specified threshold 't', for medium confidence predictions, retaining the first two predictions of the model, and determining the optimal threshold 't' based on the data of the validation group by considering indicators including the proportion of high confidence predictions, the accuracy of high confidence predictions, and the accuracy of medium confidence predictions. Select the average value with the highest threshold 't';

[0069] Step S502, model evaluation and statistical analysis, continuous variables are expressed as medians (ranges), and categorical variables are expressed as frequencies (percentages). Continuous data are analyzed using Student's t test, and categorical data are analyzed using chi-square test. The diagnostic accuracy of the model is evaluated using balanced accuracy, accuracy, and the area under the receiver operating characteristic curve. The DeLong test is used to determine the 95% confidence interval of the area under the curve and compare the differences between different areas under the curve to obtain the optimal discrimination model.

[0070] like Figure 2 As shown, the balance accuracy is used as the indicator for classifier evaluation, where:

[0071] A, Balanced accuracy of the classifier using deep features extracted from cubes with different imaging periods and segmentation strategies containing background information.

[0072] B. Balanced accuracy of the classifier using deep features extracted from cubes with different imaging periods and segmentation strategies without background information.

[0073] C, Balanced accuracy of classifiers using radiomic features extracted from cubes with different imaging periods and segmentation strategies.

[0074] like Figure 3 Figure 2 shows the performance of the multi-stage classifier in renal lesion classification in the entire validation cohort and the small renal lesion cohort. The following table shows the distribution of the entire (A) and small renal lesion (D) training / validation cohorts using unified manifold approximation and projection (UMAP); the classification ROC curves of the entire (B) and small renal lesion (E) validation cohorts; and the confusion matrices of the entire (C) and small renal lesion (F) validation cohorts.

[0075] like Figure 4As shown, the final cohort included 2079 patients with pathologically confirmed renal lesions on CT images who underwent nephrectomy. Among them, 1506 (72%) were diagnosed with ccRCC, 101 (5%) were chRCC, and 53 (3%) were pRCC. 419 (20%) were benign renal lesions, including AML (324, 77%), cysts (53, 13%), renal cell tumors (21, 5%), and other uncommon benign renal tumors (21, 5%). The median age of the total cohort was 55 years (range: 18-84 years), 63% (n=1311) were male, 53% (n=1097) were located on the right side, and 58% (n=1213) were small renal lesions.

[0076] The present invention has a novel structure, ingenious conception, and simple and convenient operation. Through this design, deep features can be well extracted, benign renal tumor subtypes can be well separated, the clinical informativeness of the model output is enhanced, and the function of combining the predicted category with clinical characteristics, genetic markers and histopathology is added to assist decision-making for cases with medium confidence predictions in clinical practice.

[0077] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for distinguishing renal tumor subtypes based on 3D deep learning, characterized in that: The following steps are involved: Step S1, obtaining patient data, and screening CT images of suitable patients with renal disease according to inclusion and exclusion criteria; Step S2, image processing and segmentation, downloading CT images of all patient data and converting their formats, and then performing image segmentation and processing; Step S3, radiomics feature extraction, after normalizing and discretizing all patient data, radiomics feature extraction is performed; Step S4: deep feature extraction to obtain deep features, and then construct a model based on the radiomics features and the deep features; Step S5: Obtain an optimal discrimination model, and use the optimal discrimination model to predict the type of renal tumor subtype.

2. The method for distinguishing renal tumor subtypes based on 3D deep learning according to claim 1, characterized in that: In step S1, the inclusion criteria include the following: Standard S101: renal lesions confirmed by histopathology to be benign, clear cell renal carcinoma, papillary renal carcinoma, or chromophobe cell carcinoma; Standard S102: Provide preoperative CT axial images that include at least the arterial and venous phases; Standard S103, CT image quality is suitable for analysis, without severe motion or artifacts.

3. The method for distinguishing renal tumor subtypes based on 3D deep learning according to claim 1, characterized in that: In step S2, the CT images of all patient data are downloaded in DICOM format with their original size and resolution, and saved in NiFTI format after anonymization.

4. The method for distinguishing renal tumor subtypes based on 3D deep learning according to claim 1, characterized in that: In step S2, the image segmentation and processing includes the following steps: Step S201, manually outlining the arterial phase image layer by layer to obtain voxel-level segmentation labels; Step S202: The voxel-level segmentation labels obtained in step S201 are manually reviewed and improved to obtain the tumor region.

5. The method for distinguishing renal tumor subtypes based on 3D deep learning according to claim 1, characterized in that: In step S4, deep feature extraction includes the following steps: Step S401, pre-training, cropping a cube containing both the ROI and background information according to the range of the ROI, filling the non-foreground area in the cropped data with zeros, and using a predefined vector to translate the lesion center before re-cropping to enhance the training data, thereby limiting overfitting; Step S402, fine-tuning, using the data of the training group for model parameter optimization, and minimizing the loss function through the Adam optimizer to identify benign and malignant renal tumor subtypes; Step S403, deep feature extraction, first, adjust the window width and window level, then use linear transformation to normalize the pixel values ​​of the image, and finally, extract the deep features from the fully connected layer before the SoftMax layer through forward propagation calculation.

6. The method for distinguishing renal tumor subtypes based on 3D deep learning according to claim 1, characterized in that: In step S5, obtaining the optimal differentiation model includes the following steps: Step S501, high confidence and medium confidence prediction, using the output probability of the model as a confidence measure, and dividing the prediction into high confidence and medium confidence according to the specified threshold t, and for the medium confidence prediction, retaining the first two predictions of the model; calculating indicators including the proportion of high confidence predictions, the accuracy of high confidence predictions and the accuracy of medium confidence predictions, determining the optimal threshold t, and selecting the average value with the highest threshold t; Step S502, model evaluation and statistical analysis, continuous variables are expressed as medians, categorical variables are expressed as frequencies, continuous data are analyzed using Student's t test, categorical data are analyzed using chi-square test, balanced accuracy, accuracy and area under the receiver operating characteristic curve are used to evaluate the diagnostic performance of the model, the DeLong test is used to determine the 95% confidence interval of the area under the curve and compare the differences between different areas under the curve to obtain the optimal discrimination model.

Images