Method for predicting small kidney cancer invasiveness based on preoperative enhanced CT deep learning technology

Abstract

The invention discloses a method for predicting small kidney cancer invasiveness based on a preoperative enhanced CT deep learning technology. The method comprises the following steps: S1, collecting and labeling a training data set; s2, training a multi-scale feature aggregation segmentation network, and training an SRCC-Former network by using the marked CT enhanced image; s3, deployment and prediction of a segmentation network: sending a conventional kidney cancer enhanced image into the trained SRCC-Former, and obtaining an invasiveness prediction result and small kidney cancer focus segmentation prediction; and S4, outputting a model result. According to the method, the pre-operative enhanced CT image is utilized, the deep learning technology is adopted, quantitative tissue characteristics are extracted, whether the small kidney cancer has invasiveness or not is judged, and more bases are provided for selection of treatment schemes of clinicians. According to the method, unnecessary operation or ablation treatment can be effectively avoided, and the treatment risk and the medical cost are reduced.

Description

Technical Field

[0001] The present invention relates to a deep learning technology based on preoperative enhanced CT, and in particular to a method for predicting the aggressiveness of small renal cancer based on the deep learning technology of preoperative enhanced CT. Background Art

[0002] With the popularization of physical examination and imaging technology, the number of small renal tumors (≤4cm) detected in all age groups has continued to increase, resulting in a significant increase in the number of surgical resections. However, the renal cancer-specific mortality rate has not decreased accordingly, indicating that many patients do not benefit from resective treatment. Data show that about 20% of small renal masses are benign tumors, mainly including leiomyolipomas and oncocytomas. Even for malignant tumors, only 12-16% are high-grade tumors that may be invasive, while most malignant tumors are indolent and will not recur or metastasize. Therefore, active monitoring of small renal tumors with benign or indolent manifestations, surgical resection or ablation of invasive small renal cancer, and preoperative differentiation between invasive and indolent renal tumors have important clinical value, which also helps to reduce unnecessary treatment-related complications and medical costs.

[0003] At present, the diagnostic accuracy of clinically distinguishing between invasive and indolent renal tumors is not high, relying solely on imaging performance to evaluate pathological grading and peripheral infiltration. Although the pathological diagnosis rate of puncture biopsy is 70-90%, due to local sampling and internal heterogeneity of the tumor, it can only indicate malignancy, but not pathological grading. With the application of artificial intelligence in medical imaging, machine vision can obtain a large number of image features that are beyond the range of human eye evaluation, providing an opportunity to quantify sub-visual tissue heterogeneity. At present, the research on machine learning of renal cancer is mainly aimed at the differential diagnosis of benign and malignant, and there are no reports on the use of artificial intelligence models based on preoperative CT images to predict the invasiveness of small renal cancer. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method for predicting the aggressiveness of small renal cancer based on preoperative enhanced CT deep learning technology.

[0005] Technical solution: The present invention provides a method for predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology, comprising the following steps:

[0006] Step S1: Collection and annotation of training data set;

[0007] Step S2: Multi-scale feature aggregation segmentation network training, using the annotated CT enhanced images to train the SRCC-Former network, such as Figure 2 As shown;

[0008] Step S3: Deployment and prediction of the segmentation network, sending the conventional renal cancer enhanced image to the trained SRCC-Former to obtain the invasive prediction results and the segmentation prediction of small renal cancer lesions;

[0009] Step S4: Output the model results.

[0010] Further, step S1: collection and annotation of training data set: approved by the Ethics Committee of Zhongshan Hospital Affiliated to Fudan University, 1587 patients with renal cancer who underwent partial or radical nephrectomy in the hospital were retrospectively included as the model training development cohort. The maximum diameter of the tumor in these patients was less than 4 cm, and all had preoperative enhanced CT. At the same time, 304 patients with small renal cancer in the First People's Hospital Affiliated to Zhejiang University were retrospectively included as an external validation cohort. The collected small renal cancers were annotated, and an automatic recognition and image segmentation system was constructed. According to the pathological results, small renal cancers were divided into two categories: indolent and invasive. Among them, indolent small renal cancers include I-II clear cell carcinoma and papillary cell carcinoma, chromophobe cell carcinoma, clear papillary renal cell carcinoma, multilocular cystic renal cell carcinoma with low malignant potential, epithelioid angiomyolipoma, and other low-grade malignant tumors such as mucinous tubular spindle cell carcinoma. Aggressive small renal cell carcinomas include clear cell carcinoma and papillary cell carcinoma with grade III-IV or sarcomatous changes, renal cell carcinoma with TEF-3 rearrangement, unclassified renal cell carcinoma, and other high-grade renal cancers such as collecting duct carcinoma.

[0011] Further, step S2: construct a multi-scale feature aggregation segmentation network for segmenting lesion areas and distinguishing between indolent and invasive small renal cancers. Select the segmentation network weights with superior performance based on the external validation set.

[0012] Furthermore, step S3: deployment and prediction of segmentation network: the trained network is deployed, and the enhanced CT image is input into the target network to obtain the segmentation result of the lesion and the invasiveness classification prediction result.

[0013] Further, step S4: outputting the model results to assist doctors in making decisions. Based on the prediction results of the model, more basis is provided for doctors to choose treatment plans.

[0014] Currently, radiologists cannot accurately determine whether small renal cell carcinoma is invasive before surgery with the naked eye. Neural network technology has been widely used in the field of medical imaging. It has stronger feature extraction and data distribution fitting capabilities. It can learn potential clinical image features that are imperceptible to the human eye, providing a reliable method for better mining tumor invasive characteristics. This study used deep learning technology to build an automatic recognition and image segmentation model based on preoperative enhanced CT images, which can classify small renal cell carcinomas as invasive. By retrospectively incorporating a large number of renal cancer patients as a model development cohort and external validation cohort, a highly accurate invasive classification prediction model was finally obtained. This study provides more accurate decision support for the clinical treatment of patients with small renal cell carcinoma.

[0015] Different clinical management methods are required for aggressive and indolent small renal cell carcinomas (≤4 cm). Currently, radiologists cannot accurately determine whether small renal cell carcinomas are invasive before surgery by naked eyes. This study used preoperative enhanced CT images and deep learning technology to extract quantitative tissue features to determine whether small renal cell carcinomas are invasive, providing more basis for clinicians to choose treatment options. This method can effectively avoid unnecessary surgery or ablation therapy, reduce treatment risks and medical expenses, and improve the quality of life of patients.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] 1. Build an automatic recognition and segmentation model to avoid manually outlining the area of ​​interest.

[0018] 2. Use deep learning technology based on Vision Transformer neural network to deeply extract information from CT images and build models to reduce image information omissions.

[0019] 3. Import conventional kidney cancer enhanced images to obtain invasiveness prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the process of the present invention;

[0021] Figure 2 Schematic diagram of the multi-scale feature aggregation segmentation network architecture;

[0022] Figure 3 This is a schematic diagram of the LGIM module. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be further described below.

[0024] A method for predicting the aggressiveness of small renal cancer based on preoperative enhanced CT deep learning technology in this embodiment includes the following steps:

[0025] Step S1: Collection and annotation of training data set. With the approval of the Ethics Committee of Zhongshan Hospital Affiliated to Fudan University, 1587 patients with small renal cell carcinoma (tumor diameter <4 cm) who underwent partial or radical nephrectomy in the hospital were retrospectively included for model training. All patients had preoperative enhanced CT. Another 304 patients with small renal cell carcinoma who underwent surgery in the First People's Hospital Affiliated to Zhejiang University were included as an external validation cohort. The collected preoperative enhanced CT images were annotated to construct an automatic recognition and image segmentation system. According to the pathological results, small renal cell carcinoma was further divided into two categories: indolent and invasive. Indolent renal cell carcinoma includes stage I-II clear cell carcinoma, papillary cell carcinoma, chromophobe cell carcinoma, multilocular cystic carcinoma with low malignant potential, etc.; invasive renal cell carcinoma includes stage III-IV or sarcomatoid clear cell carcinoma, TEF-3 rearranged gene renal carcinoma, unclassified and collecting duct carcinoma, etc.

[0026] Step S2: Multi-scale feature aggregation segmentation network training.

[0027] 1) Network training process:

[0028] like Figure 2 As shown in the figure, during the training process, the annotated preoperative enhanced CT image i is input into the designed multi-scale encoder PVT encoder. The encoder adopts the PVT architecture based on Vision Transformer and consists of four layers of pyramid structure encoding layers. For each encoding layer, features F = {X1, X2, X3, X4} of different scales and feature dimensions are extracted from shallow to deep. The first encoding layer features Among them, H and W are the input image sizes, and 64 is the number of feature channels. After that, the spatial scale of the features of each coding layer is halved and the feature dimension is doubled, so as to extract feature expressions of different knowledge granularity and dimensions.

[0029] Then, the multi-scale features F are fed into the designed feature fusion module to fully mine information of different granularities. Figure 2 As shown in the figure, for the last three layers of encoded features {X2, X3, X4}, they are first sent to the designed local global interaction module (LGIM) and a layer of 1x1 convolution module. The schematic diagram of the LGIM module structure is shown in the figure. Figure 3 As shown in the figure, it further mines the multi-scale information of each layer of encoded features and perceives the small renal cancer lesion area in a fine-grained manner. The 1x1 convolution module consists of a 1x1 convolution layer, a Batch Norm (BN) layer, and a ReLu activation layer, which is responsible for upsampling the encoded features {X2, X3, X4} of different spatial sizes to the same feature scale. For the first layer of encoded features X1, only one layer of 1x1 convolutional module is used to upsample its feature scale to the same The characteristic scale is consistent, and we get After that, a High-low-level fusion module (HLFM) is designed to combine adjacent features Aggregate and obtain the final fusion feature X containing multi-scale information.

[0030] Finally, the fusion feature X is sent to a segmentation head (Seg Head) and a classification head (Cls Head) to perform segmentation prediction of the lesion area and classification of the pathology into inertia and invasiveness, respectively, to obtain the segmentation prediction P s And the classification prediction P c Among them, the DeepLab-V2 segmentation head structure is adopted, and the binary cross entropy (BCE) loss function and the intersection over union (IoU) loss function are used for supervision; the classification head uses a linear layer for prediction and uses the binary cross entropy (BCE) loss function for supervision. The loss function of this network is expressed as follows:

[0031] L=L BCE (P s , Y s )+γL IoU (P s , Y s )+λL BCE (P c , Y c )

[0032] Among them, Y s is the segmentation mask true label, Y c is the classification label. γ and λ are loss coefficients, which are used to balance the contribution weights of multiple losses.

[0033] 2) Introduction to key modules:

[0034] HLFM module: Figure 2 As shown in Figure 2, the HLFM module is responsible for aggregating the features of adjacent coding layers to achieve the interaction between high-level semantic features and low-level semantic features, thereby deeply mining smaller lesions. as well as Take as an example, and input them into a 1x1 convolution module respectively, and get as well as Then add the two together to get After the sigmoid activation function is activated, the activated features are added again and sent to a 3x3 convolution module to obtain The process is described as:

[0035]

[0036] Among them, Conv(.) represents a 3x3 convolution module, and α(.) is the sigmoid activation function.

[0037] Then, and Obtained by HLFM polymerization Will and The final fusion feature X is obtained through HLFM aggregation.

[0038] LGIM module: Figure 3 As shown in Figure 1, the LGIM model is responsible for interactive learning of local and global information, thereby achieving the complementarity of the two information, which is beneficial to the identification and segmentation of lesions. Specifically, for the input X l , 1 = {2, 3, 4}. Separate it along the channel direction to obtain 4 sub-features For each sub-feature, convolution kernels of different sizes, 1x1, 3x3, and 5x5, are used to capture feature information of different granularities. Among them, the 1x1 convolution module has the smallest receptive field and extracts the most local fine-grained knowledge. In order to obtain global feature representation, GAP (Global Average Pooling) is used to extract global semantic information. Then, the features of different granularities are obtained. It is cascaded again along the feature channel direction and sent to a 1x1 convolution module and a GELU activation function for activation to obtain the feature x l . A residual connection is set up to convert x l With X l Add them together to get the final output features

[0039] After the network training was completed, a large number of renal cancer patients were retrospectively included as the model development cohort and external validation cohort, and finally a highly accurate invasive classification prediction model was obtained. This study provides more accurate decision support for the clinical treatment of patients with small renal cancer.

[0040] Step S3: Deployment and prediction of segmentation network: The trained network is deployed, and the enhanced CT image is input into the target network to obtain the segmentation result of the lesion and the invasiveness classification prediction result.

[0041] Step S4: Output the model results to assist doctors in making decisions. Based on the model's prediction results, provide more basis for doctors to choose treatment plans

[0042] The acquired CT images (including the plain scan phase, arterial phase, and venous phase) are input into the small renal cell carcinoma invasiveness prediction model of the present invention, and after analysis, the prediction result is outputted that the small renal cell carcinoma is invasive or indolent.

[0043] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.

Claims

1. A method for predicting the aggressiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology, characterized in that: The steps include: Step S1: Collection and annotation of training data set; Step S2: Multi-scale feature aggregation segmentation network training, using the annotated CT enhanced images to train the SRCC-Former network; Step S3: Deployment and prediction of the segmentation network, sending the conventional renal cancer enhanced image to the trained SRCC-Former network to obtain the invasiveness prediction results and the segmentation prediction of small renal cancer lesions; Step S4: Output the model results.

2. The method for predicting the aggressiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 1, characterized in that: The step S1: retrospectively enroll patients with small renal cancer for model training, all patients have preoperative enhanced CT, and also enroll patients with small renal cancer as an external validation cohort, annotate the collected enhanced CT of renal cancer, and construct an automatic recognition and image segmentation system.

3. The method for predicting the aggressiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 2, characterized in that: The network training process in step S2 is as follows: the annotated preoperative enhanced CT image is input into the multi-scale encoder PVT encoder to extract feature expressions of different knowledge granularities and scales, and then the multi-scale features are sent to the feature fusion module to mine information of different granularities.

4. The method for predicting the aggressiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 3, characterized in that: The encoder adopts the PVT architecture based on Vision Transformer, which includes four layers of pyramid structure coding layers. For each coding layer, features of different scales and feature dimensions are extracted from shallow to deep. F = {X1, X2, X3, X4}, the first coding layer features Among them, H and W are the input image sizes, and 64 is the number of feature channels. After that, the spatial scale of the features of each coding layer is halved and the feature dimension is doubled to extract feature expressions of different knowledge granularity and scale.

5. The method for predicting the aggressiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 4, characterized in that: For the last three layers of encoded features {X2, X3, X4}, they are first sent to the local-global interaction module LGIM and a 1x1 convolution module respectively to further mine the multi-scale information of each layer of encoded features and perceive small renal cancer lesions in a fine-grained manner.

6. The method for predicting the aggressiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 5, characterized in that: The 1x1 convolution module consists of a 1x1 convolution layer, a Batch Norm layer, and a ReLu activation layer, which is responsible for upsampling the coded features {X2, X3, X4} with different spatial sizes to the same feature scale.

7. The method for predicting the aggressiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 6, characterized in that: For the first layer of encoded features X1, a 1x1 convolution module is used to scale its features Upsampled to Consistent with the scale, and obtained After that, the High-low-level fusion module HLFM is used to combine adjacent features Aggregate and obtain the final fusion feature X containing multi-scale information.

8. The method for predicting the aggressiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 7, characterized in that: The fusion feature X is sent to a segmentation head and a classification head respectively to perform segmentation prediction of the lesion area and classification of the pathology into inertia and invasiveness, and the segmentation prediction P is obtained. s And the classification prediction P c .

9. The method for predicting the aggressiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 8, characterized in that: The DeepLab-V2 segmentation head structure is adopted, and the binary cross entropy BCE loss function and the intersection over union (IoU) loss function are used for supervision; the classification head uses a linear layer for prediction and uses the binary cross entropy BCE loss function for supervision. The loss function is expressed as follows: L=L BCE (P s ,AND s )+γL IoU (P s ,AND s )+λL BCE (P c ,AND c ) Among them, Y s is the segmentation mask true label, Y c is the classification label, γ and λ are loss coefficients, which are used to balance the contribution weights of multiple losses.

10. The method for predicting the aggressiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 9, characterized in that: The high-low-level fusion module HLFM is responsible for aggregating the features of adjacent coding layers to achieve the interaction between high-level semantic features and low-level semantic features, and then deeply mine smaller lesions; the local-global interaction module LGIM is responsible for interactive learning of local and global information, so as to achieve the complementarity of the two information, which is conducive to the identification and segmentation of lesions.

Images