A method for predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology

Abstract

The application discloses a method for predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology, and steps include: step S1, collection and labeling of training data set; step S2, multi-scale feature aggregation segmentation network training, using the labeled CT enhanced image to train the SRCC-Former network; step S3, deployment and prediction of the segmentation network, sending the conventional renal carcinoma enhanced image into the trained SRCC-Former to obtain the invasiveness prediction result and small renal cell carcinoma lesion segmentation prediction; and step S4, outputting the model result. The application uses the preoperative enhanced CT image, adopts the deep learning technology, extracts the quantitative tissue features, judges whether the small renal cell carcinoma has invasiveness or not, and provides more basis for the selection of a treatment scheme for a clinician. The method can effectively avoid unnecessary surgery or ablation treatment, and reduce treatment risk and medical expenses.

Description

TECHNICAL FIELD

[0001] The application relates to a preoperative enhanced CT deep learning technology, in particular to a method for predicting invasiveness of small renal cell carcinoma based on the preoperative enhanced CT deep learning technology. BACKGROUND

[0002] With the popularization of physical examination and imaging technology, the number of small renal tumors (<=4cm) detected in each age group is increasing, leading to a significant increase in the number of surgical resection. However, the specific mortality rate of renal cell carcinoma has not decreased accordingly, which indicates that many patients do not benefit from resection therapy. Data shows that about 20% of small renal tumors are benign tumors, mainly including leiomyolipoma and oncocytoma. Even if it is a malignant tumor, only 12-16% is a high-grade tumor that may be invasive, while most malignant tumors show inertia and do not relapse or metastasize. Therefore, it has important clinical value to actively monitor small renal tumors with benign or inert performance, surgically or ablatively invasive small renal cell carcinoma, and to distinguish invasive and inert renal tumors before surgery, which also helps to reduce unnecessary treatment-related complications and medical costs.

[0003] Currently, the clinical diagnosis accuracy of distinguishing invasive and inert renal tumors is not high by relying only on imaging performance to evaluate pathological grading and peripheral infiltration. Although the pathological diagnosis rate of puncture biopsy is 70-90%, due to local sampling and tumor internal heterogeneity differences, it can only suggest malignancy and cannot well suggest pathological grading. With the application of artificial intelligence in medical images, machine vision can obtain a large number of image features beyond the assessable range of the human eye, providing an opportunity for quantifying sub-visual tissue heterogeneity. At present, the machine learning research of renal cell carcinoma is mainly aimed at benign and malignant differential diagnosis, and there is no report on the artificial intelligence model based on preoperative CT images to predict the invasiveness of small renal cell carcinoma. SUMMARY

[0004] The application aims to provide a method for predicting invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology.

[0005] Technical scheme: The application provides the method for predicting invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology, which comprises the following steps:

[0006] Step S1: collection and labeling of training data set;

[0007] Step S2: multi-scale feature aggregation segmentation network training, using the labeled CT enhanced image to train the SRCC-Former network, as shown in Figure 2 .

[0008] Step S3: Deployment and prediction of the segmentation network, send the enhanced image of the conventional renal cancer into the trained SRCC-Former to obtain the prediction results of invasiveness and the segmentation prediction of small renal cancer lesions;

[0009] Step S4: Output model results.

[0010] Further, step S1: collection and annotation of training data set: after approval by the ethics committee of Zhongshan Hospital, Fudan University, 1587 patients with renal cancer who underwent partial or radical nephrectomy in the hospital were retrospectively included as model training development cohort. The maximum diameter of the tumor of these patients is less than 4cm, and they all have preoperative enhanced CT. At the same time, 304 cases of small renal cancer in the First People's Hospital of Zhejiang University were retrospectively included as external verification cohort. The collected small renal cancers were labeled, and an automatic identification and image segmentation system was constructed. According to the pathological results, small renal cancers were divided into two categories: indolent and invasive. Among them, indolent small renal cancer includes I-II clear cell carcinoma and papillary cell carcinoma, chromophobe cell carcinoma, clear cell papillary renal cell carcinoma, low malignant potential multilocular cystic renal cell carcinoma, epithelioid hemangiosarcoma, and other low malignant tumors such as mucinous tubular and spindle cell carcinoma. Invasive small renal cancer includes III-IV or sarcomatoid clear cell carcinoma and papillary cell carcinoma, TEF-3 rearranged gene renal cell carcinoma, unclassified renal carcinoma, and other high-grade renal carcinomas such as collecting duct carcinoma, etc.

[0011] Further, step S2: construct a multi-scale feature aggregation segmentation network. Used for segmenting the lesion area and distinguishing indolent and invasive small renal cancer. According to the external verification set, the segmentation network weight with superior performance is selected.

[0012] Further, step S3: deployment and prediction of the segmentation network. The trained network is deployed, and the enhanced CT image is input into the target network to obtain the segmentation result of the lesion and the prediction result of invasiveness classification.

[0013] Further, step S4: output model results to assist doctors in decision-making. According to the prediction results of the model, more evidence is provided for doctors to choose treatment options.

[0014] Currently, radiologists cannot accurately determine whether small renal cell carcinoma is invasive or not by naked eye. Neural network technology has been widely used in the field of medical images, with stronger feature extraction and data distribution fitting capabilities, and can learn potential clinical image features that the human eye cannot detect, providing a reliable method for better mining of tumor invasive features. In this study, a deep learning technology was used to construct an automatic identification and image segmentation model based on preoperative enhanced CT images, which can classify small renal cell carcinoma as invasive or not. By retrospectively including a large number of renal cancer patients as model development and external validation cohorts, a high-accuracy invasive classification prediction model was finally obtained. This study provides more accurate decision support for the clinical treatment of small renal cell carcinoma patients.

[0015] Invasive and indolent small renal cell carcinoma (≤4cm) correspond to different clinical management. Currently, radiologists cannot accurately determine whether small renal cell carcinoma is invasive or not by naked eye. In this study, preoperative enhanced CT images were used to extract quantitative tissue features using deep learning technology to determine whether small renal cell carcinoma is invasive or not, providing more evidence for clinicians to choose treatment options. This method can effectively avoid unnecessary surgery or ablation therapy, reduce treatment risks and medical costs, and improve patient quality of life.

[0016] Compared with the prior art, the present application has the following beneficial effects:

[0017] 1. An automatic identification and segmentation model is constructed to eliminate the need for manual outlining of the region of interest.

[0018] 2. Deep learning technology based on Vision Transformer neural network is used to extract information in CT images and construct models, reducing the omission of image information.

[0019] 3. Importing conventional renal cancer enhanced images, the invasive prediction result is obtained. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The present application is a method flowchart;

[0021] Figure 2 It is a multi-scale feature aggregation segmentation network architecture schematic diagram;

[0022] Figure 3 It is a LGIM module schematic diagram. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be further described below.

[0024] A method for predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to the present embodiment comprises the following steps:

[0025] Step S1: Collection and annotation of training dataset. With the approval of the Ethics Committee of Zhongshan Hospital, Fudan University, 1587 patients with small renal cell carcinoma (tumor diameter <4 cm) who underwent partial or radical nephrectomy in this hospital were retrospectively included for model training, and all patients had preoperative contrast-enhanced CT. Another 304 patients with small renal cell carcinoma who underwent surgery in the First People's Hospital of Zhejiang University were included as an external validation cohort. The collected preoperative contrast-enhanced CT images were annotated to construct an automatic identification and image segmentation system. According to the pathological results, small renal cell carcinoma was further divided into two categories: indolent and invasive. Indolent renal cell carcinoma includes stage I-II clear cell carcinoma, papillary cell carcinoma, chromophobe cell carcinoma, and low-grade potential multi-locular cystic carcinoma; invasive renal cell carcinoma includes stage III-IV or sarcomatoid clear cell carcinoma, TEF-3 rearranged gene renal carcinoma, unclassified, and collecting duct carcinoma, etc.

[0026] Step S2: Training of multi-scale feature aggregation segmentation network.

[0027] 1) Network training process:

[0028] As shown in Figure 2 , during the training process, the annotated preoperative contrast-enhanced CT image i is input into the designed multi-scale encoder PVT encoder. The encoder uses the PVT architecture based on Vision Transformer, which includes four layers of pyramid structure encoding layers. For each encoding layer, different scales and feature dimensions of features F = {X1, X2, X3, X4} are extracted from shallow to deep. where H and W are the input image size, and 64 is the feature channel number. After that, the feature of each encoding layer is halved in spatial scale and doubled in feature dimension, so as to extract features of different knowledge granularity and dimension.

[0029] Subsequently, the multi-scale features F are sent to the designed feature fusion module (Fusion module) to fully exploit the information of different granularities. As shown in Figure 2 , for the last three layers of encoding features {X2, X3, X4}, they are first sent to the designed local-global interaction module (LGIM) and a 1x1 convolution module. The LGIM module structure diagram is shown in Figure 3 , which further extracts the multi-scale information of each encoding feature and perceives the small renal cell carcinoma lesion area in detail. The 1x1 convolution module is composed of a 1x1 convolution layer, a Batch Norm (BN) layer, and a ReLu activation layer, which is responsible for upsampling different spatial size encoding features {X2, X3, X4} to the same feature scale to obtain For the first layer of encoded features X1, only one layer of 1x1 convolution module is used to upsample its feature scale to be consistent with the feature scale of other layers, obtaining After that, a High-low-level fusion module (HLFM) is designed to aggregate adjacent features and obtain the final fusion features X containing multi-scale information.

[0030] Finally, the fusion features X are sent into a segmentation head (Seg Head) and a classification head (Cls Head) respectively to perform lesion region segmentation prediction and pathological indolence and invasiveness classification, obtaining the segmentation prediction P s and the classification prediction P c . The DeepLab-V2 segmentation head structure is used, and the binary cross-entropy (BCE) loss function and the intersection over union (IoU) loss function are used for supervision; the classification head uses a linear layer for prediction and the binary cross-entropy (BCE) loss function for supervision. The loss function of the network is expressed as follows:

[0031] L = L BCE (P s , Y s ) + γL IoU (P s , Y s ) + λL BCE (P c , Y c )

[0032] where Y s is the segmentation mask true label, Y c is the classification label. γ and λ are loss coefficients to balance the contribution weight of multiple losses.

[0033] 2) Key module introduction:

[0034] HLFM module: as shown in Figure 2 , the HLFM module is responsible for aggregating adjacent encoded layer features to realize the interaction of high-level semantic features and low-level semantic features, and thus to deeply mine small lesions. Taking features and as examples, they are respectively input into a 1x1 convolution module to obtain and Then, the two are added to obtain After sigmoid activation function activation, the activated features are added again and sent into a 3x3 convolution module to obtain The process is described as:

[0035]

[0036] wherein Conv(.) represents a 3x3 convolution module, and a(.) is a sigmoid activation function.

[0037] Then, the following is obtained: and The final fusion feature X is obtained by HLFM aggregation. The following is obtained: and The final fusion feature X is obtained by HLFM aggregation.

[0038] The LGIM module: as shown in the following formula, the LGIM module is responsible for interactive learning of local and global information, so as to realize complementation of the information of the two, which is conducive to identification and segmentation of the lesion. Figure 3 Specifically, for the input X l , 1 = {2, 3, 4}. Separation is performed along the channel direction to obtain four sub-features For each sub-feature, 1x1, 3x3 and 5x5 different size convolution kernels are used to capture feature information of different granularities. Among them, the 1x1 convolution module has the smallest receptive field and extracts the most local fine-grained knowledge. In order to obtain global feature representation, GAP (Global Average Pooling) global average pooling is used to extract global semantic information. Then, the obtained different granularity features are cascaded along the feature channel direction and input into a 1x1 convolution module and a GELU activation function for activation to obtain the feature x l . A residual connection is set to add x l and X l to obtain the final output feature

[0039] After the network is trained, a large number of kidney cancer patients are retrospectively included as a model development queue and an external verification queue, and finally an accurate invasive classification prediction model is obtained. The research provides more accurate decision support for the clinical treatment of small kidney cancer patients.

[0040] Step S3: deployment and prediction of the segmentation network. The trained network is deployed, and the enhanced CT image is input into the target network to obtain the segmentation result of the lesion and the invasive classification prediction result.

[0041] Step S4: output model results to assist doctors in decision-making. According to the prediction result of the model, more basis is provided for the selection of the treatment plan of the doctor

[0042] The CT image (including the plain scan period, the arterial phase and the venous phase) is input into the small kidney cancer invasive prediction model of the application, and the prediction result that the small kidney cancer is invasive or inert is output after analysis.

[0043] The above merely describes the preferred embodiments of the present application, and does not limit the present application in any way. Any person skilled in the art, without departing from the scope of the technical solutions of the present application, can make any form of equivalent replacement or modification of the technical solutions and technical contents disclosed by the present application, and such changes still belong to the protection scope of the present application.

Claims

1. A method for predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology, characterized in that, The method comprises the following steps: Step S1: collection and labeling of training data set; Step S2: training of multi-scale feature aggregation segmentation network, using the labeled CT enhanced image to train the SRCC-Former network, The SRCC-Former comprises: A PVT encoder based on Vision Transformer, which comprises four layers of pyramid structure encoding layers for extracting features of different knowledge granularity and scale; A local-global interaction module LGIM, which performs 1x1 convolution, 3x3 convolution, 5x5 convolution and global average pooling on the encoding features of the last three layers respectively in parallel to capture local texture and global context; A high-low-level fusion module HLFM responsible for aggregating adjacent encoding layer features to realize the interaction of high-level semantic features and low-level semantic features; Step S3: deployment and prediction of the segmentation network, inputting the conventional renal carcinoma enhanced image into the trained SRCC-Former network to obtain the invasiveness prediction result and small renal carcinoma lesion segmentation prediction; Step S4: outputting the model result, LGIM module: The LGIM module is responsible for the interaction learning between local and global information. Specifically, for the input , = {2, 3, 4}, the feature is separated along the channel direction to obtain four sub-features . For each sub-feature, different granularity feature information is captured by using 1x1, 3x3 and 5x5 convolution kernels. The receptive field of the 1x1 convolution module is the smallest, and the most local fine-grained knowledge is extracted. In order to obtain global feature representation, GAP global average pooling is used to extract global semantic information. Then, the different granularity features obtained are . The feature is cascaded along the feature channel direction again and sent to a 1x1 convolution module and a GELU activation function for activation to obtain the feature . A residual connection is set to add and to obtain the final output feature .

2. The method of predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 1, characterized in that, The step S1: retrospectively including small renal carcinoma patients for model training, all patients have preoperative enhanced CT, and another small renal carcinoma patient is included as an external validation queue, the collected renal carcinoma enhanced CT is labeled, and an automatic identification and image segmentation system is constructed.

3. The method of predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 2, characterized in that, The network training process in step S2: input the labeled preoperative enhanced CT image into the multi-scale encoder PVT encoder to extract features of different knowledge granularity and scale, then send the multi-scale features to the feature fusion module Fusion module to mine information of different granularity.

4. The method of predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 3, characterized in that, Each layer of the encoding layer, from shallow to deep, extracts features of different scales and feature dimensions , the first layer of the encoding layer feature , where H and W are the input image size, 64 is the number of feature channels, and thereafter, the feature of each layer of the encoding layer is halved in spatial scale and doubled in feature dimension, extracting features of different knowledge granularity and scale.

5. The method of predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 4, characterized in that, For the last three layers of encoding features , they are first sent into the local-global interaction module LGIM and a 1x1 convolution module respectively, to further mine the multi-scale information of each layer of encoding features and perceive the small renal cell carcinoma lesion region in fine granularity.

6. The method of predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 5, characterized in that, The 1x1 convolution module is composed of a 1x1 convolution layer, a Batch Norm layer and a ReLu activation layer, responsible for encoding features with different spatial sizes upsampling to the same feature scale, obtaining .

7. The method of predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 6, characterized in that, Encoding features for the first layer are reduced to a scale of 1x1 upsampling to be consistent with and get After that, the adjacent features are aggregated and the fusion features containing multi-scale information are obtained .

8. The method of predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 7, characterized in that, The fused features are respectively sent into a segmentation head and a classification head, to respectively perform lesion region segmentation prediction and pathological indolence and invasiveness classification, to obtain segmentation prediction and classification prediction .

9. The method of predicting the invasiveness of small renal cell carcinoma based on preoperative enhanced CT deep learning technology according to claim 8, characterized in that, Adopt DeepLab-V2 segmentation head structure, and adopt binary cross entropy BCE loss function And intersection over union IoU loss function Supervision; The classification head uses a linear layer for prediction and a binary cross-entropy (BCE) loss function With supervision, the loss function is expressed as follows: + + ； wherein, is a segmentation mask ground truth, is a classification label, and is a loss coefficient to balance the contribution weight of the plurality of losses.

Images