Non-functional pancreatic neuroendocrine tumor preoperative index prediction method and system

Abstract

The invention belongs to the technical field of medical image processing, and particularly relates to a non-functional pancreatic neuroendocrine tumor preoperative index prediction method and system. According to the method, on the basis of abdomen CT image lesion segmentation, pathological grading (low risk / high risk) and lymph node metastasis (LNM) risks are jointly predicted through a multi-task deep learning framework; according to the method, tumor morphological features, radiomics features and clinical parameters (such as tumor size and position) extracted by a segmentation network are combined, and high-precision prediction of preoperative key indexes is realized by using a feature fusion module and a lightweight classification head; through a cross-modal attention mechanism and multi-center data verification, the model achieves the pathologic classification AUC of 0.75 on an internal verification set, achieves the LNM prediction AUC of 0.78, and is significantly superior to traditional clinical experience judgment. According to the method, reliable decision support is provided for preoperative precise operation planning and personalized treatment.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a method and system for predicting preoperative indicators of non-functional pancreatic neuroendocrine tumors. Background Art

[0002] Accurate preoperative evaluation of non-functional pancreatic neuroendocrine tumors (NF-PNETs) is a core part of clinical decision-making. Accurate preoperative assessment of lymph node metastasis (LNM) and pathological grading is crucial for guiding lymph node resection and the extent of surgical resection to optimize treatment decisions.

[0003] Current surgical decisions rely heavily on empirical judgment, which is prone to errors and can lead to adverse postoperative outcomes, including complications such as pancreatic fistula. Preoperative management of NF-PNETs requires simultaneous assessment of pathological grading (low-risk / high-risk) and lymph node metastasis (LNM) risk. Low-risk includes G1 stage, and high-risk includes G2 and G3 stages. Traditional methods suffer from a lag in pathological grading and rely on postoperative pathological examination, failing to provide real-time guidance on the extent of intraoperative lymph node dissection. Furthermore, empirical judgment based on CT morphological features (such as tumor boundary irregularity and enhancement heterogeneity) has low sensitivity (approximately 60%) and is subject to inter-observer variability. Multimodal data suffers from fragmentation, with existing prediction models often independently training segmentation and classification tasks. These models fail to effectively integrate tumor morphological features, radiomics (such as texture and grayscale distribution), and clinical parameters (such as tumor size and location), resulting in insufficient information utilization. Furthermore, black-box models struggle to locate key regions that influence prediction outcomes, reducing physicians' trust in AI-based results.

[0004] There is an urgent need for a preoperative indicator prediction method and system that can infer reliable prediction results of pathological grading and LNM risk assessment based on segmentation results.

[0005] This method combines tumor morphological features extracted by a segmentation network with radiomics features and clinical parameters (such as tumor size and location), utilizing a feature fusion module and a lightweight classification head to achieve high-precision prediction of key preoperative indicators, overcoming the limitations of traditional techniques. It provides efficient and reliable technical support for the preoperative indicator prediction of NF-PNETs, ​​and provides reliable decision-making support for precise preoperative surgical planning and personalized treatment. Summary of the Invention

[0006] The purpose of the present invention is to provide a method and system for automatic segmentation of non-functional pancreatic neuroendocrine tumor lesions with high prediction accuracy and good reliability, so as to provide decision support for precise preoperative surgical planning and personalized treatment.

[0007] The application provides a preoperative index prediction method for non-functional pancreatic neuroendocrine tumors, which is realized based on medical image processing, and the specific steps are as follows:

[0008] (1) processing the medical image; including pre-processing the input abdominal CT image, and performing lesion segmentation on the abdominal CT image based on an improved U-Net model to obtain a segmentation mask and morphological features of the tumor region;

[0009] (2) extracting radiomics features from the segmentation mask, including texture features (such as contrast and energy of a gray level co-occurrence matrix), shape features (such as volume and sphericity), and gray level distribution features (mean and variance of CT values);

[0010] (3) fusing the morphological features, radiomics features and clinical parameters (such as the maximum diameter of the tumor, the location of the head / body / tail of the pancreas, and the age of the patient), and generating multi-modal fusion features through a cross-modal attention mechanism;

[0011] (4) using a multi-task classifier to jointly predict the pathological grade (low risk / high risk) and the risk probability of lymph node metastasis, and optimizing the classification results through a dynamic threshold adjustment strategy.

[0012] Further:

[0013] In step (1), the pre-processing of the input medical image includes inputting an abdominal enhanced CT image (such as an arterial phase and a portal phase), isotropic resampling (0.7x0.7x1mm 3 ) and window width / window level adjustment (250 / 50HU); the input clinical parameters include the maximum diameter of the tumor (unit: cm), the anatomical location (head / body / tail of the pancreas), the age of the patient, and the gender;

[0014] The lesion segmentation includes using an improved U-Net model (containing position embedding and signed distance field supervision) to segment the tumor region, and outputting a binary mask and morphological features (such as tumor volume and surface area).

[0015] In step (2), the extraction of radiomics features includes:

[0016] Texture features: calculating contrast, energy and homogeneity through a gray level co-occurrence matrix (GLCM);

[0017] Shape features: calculating tumor volume, surface area and sphericity;

[0018] Gray features: extracting the mean, variance and kurtosis of the CT value histogram.

[0019] In step (3), the multimodal fusion feature is generated by the cross-modal attention mechanism. Specifically, the imaging omics features (dimension 50) and the morphological features (dimension 32) are spliced ​​into the imaging feature matrix F img (dimension 82), clinical parameter vector F clin (Dimension 5) is input after standardization; F is transformed into img and F clin Mapped to query (Query), key (Key), and value (Value) space.

[0020] Image feature matrix F img and clinical parameters F clin Perform linear transformation and calculate the attention weight matrix A:

[0021]

[0022] Among them, W Q ,W K is the trainable weight matrix, d is the feature dimension;

[0023] The attention weight is combined with the clinical parameter projection value F clin W V After element-by-element multiplication, add it to the image feature to obtain the fusion feature F fused :

[0024] F fused =F img +A⊙(F clin W V ) (2)

[0025] In step (3), the multi-task classifier includes:

[0026] Pathology grade prediction head: Input F fused At the fully connected layer, Softmax outputs the low-risk / high-risk probability; low-risk includes G1 period, and high-risk includes G2 and G3 periods; the loss function uses weighted cross entropy loss to alleviate class imbalance (G1 accounts for 45%, G2 accounts for 50%, and G3 accounts for 5%).

[0027] LNM prediction head: input F fused At the fully connected layer, Sigmoid outputs the risk probability of lymph node metastasis; the loss function uses focal loss (FocalLoss) to suppress the weight of easily classified samples;

[0028] Joint loss function: cross entropy loss (classification) + focal loss (LNM).

[0029] Further, the method further comprises a dynamic threshold adjustment strategy, specifically setting an initial LNM risk threshold according to preoperative clinical guidelines (such as tumor size ≥2cm), and dynamically optimizing the threshold based on the ROC curve of the validation set to balance sensitivity and specificity.

[0030] The application also provides a preoperative index prediction system for non-functional pancreatic neuroendocrine tumors based on the above method, specifically comprising: a medical image processing module, an imageomics feature extraction module, a multi-modal feature fusion module, a multi-task classification module, and a visualization platform: wherein the data preprocessing module, the imageomics feature extraction module, the multi-modal feature fusion module and the multi-task classification module respectively perform the operations of the four steps of the prediction method; the visualization platform outputs an interpretable heat map and labels the model decision key area.

[0031] Further, the visualization platform generates a heat map through class activation mapping (CAM) technology, locates the tumor area affecting the classification result, and displays it in association with clinical parameters (such as tumor edge irregularity).

[0032] The application also includes a computer-readable storage medium, which stores program code for executing the preoperative index prediction method for non-functional pancreatic neuroendocrine tumors.

[0033] Compared with the prior art, the application combines the tumor morphological features extracted by the segmentation network, the imageomics features and the clinical parameters (such as tumor size and location), uses the feature fusion module and the lightweight classification head to realize high-precision prediction of preoperative key indicators, which is significantly better than the results of traditional clinical experience judgment, provides an efficient and reliable method for precise diagnosis and treatment of non-functional pancreatic neuroendocrine tumors, provides reliable decision support for preoperative precise surgical planning and individualized treatment, and has significant clinical application value and commercialization potential.

[0034] The application has a pathology grading AUC of 0.75 and an LNM prediction AUC of 0.78 on the internal validation set, which is significantly better than the traditional clinical experience judgment. The application provides reliable decision support for preoperative precise surgical planning and individualized treatment. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The application also provides a preoperative index prediction system for non-functional pancreatic neuroendocrine tumors based on the above method, specifically comprising: a medical image processing module, an imageomics feature extraction module, a multi-modal feature fusion module, a multi-task classification module, and a visualization platform: wherein the data preprocessing module, the imageomics feature extraction module, the multi-modal feature fusion module and the multi-task classification module respectively perform the operations of the four steps of the prediction method; the visualization platform outputs an interpretable heat map and labels the model decision key area.

[0036] Figure 2Fig. 4 shows the performance evaluation of the preoperative index prediction method of non-functional pancreatic neuroendocrine tumor of the present application in the multi-task model for tumor grading and LNM prediction; wherein, a shows the receiver operating characteristic (ROC) curve of the binary tumor grading of the training cohort and the four validation cohorts (internal test, internal validation, Beijing and Guangzhou centers); b is the ROC curve showing the prediction performance of the model for LNM detection in the same cohort; c shows the quantitative diagnostic performance indicators for tumor grading and LNM prediction, including accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV), with corresponding 95% confidence intervals in all evaluated cohorts; d shows the comparison of the diagnostic performance of LNM prediction between the proposed model and the clinical guidelines with a threshold of 2 cm tumor size. DETAILED DESCRIPTION

[0037] The present application will be described in detail below with specific embodiments. The following examples will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present application. These are within the scope of protection of the present application.

[0038] Specifically, a preoperative index prediction method for non-functional pancreatic neuroendocrine tumor, as shown in Fig. 1, the method comprises the following steps: Figure 1

[0039] Input data and pre-process, perform lesion segmentation on abdominal CT images based on an improved U-Net model, and obtain segmentation masks and morphological features of tumor regions;

[0040] Extract radiomic features from the segmentation masks, including texture features (contrast and energy of gray level co-occurrence matrix), shape features (volume, sphericity) and gray level distribution features (mean and variance of CT values);

[0041] Fuse the morphological features, radiomic features and clinical parameters (maximum diameter of tumor, location of head / body / tail of pancreas, patient age), and generate multi-modal fusion features through a cross-modal attention mechanism;

[0042] Use a multi-task classifier to jointly predict pathological grading (low risk / high risk) and lymph node metastasis risk probability, and optimize the classification results through a dynamic threshold adjustment strategy.

[0043] First, input the image I and output I' after pre-processing, including isotropic resampling (0.7x0.7x1mm 3 ) and window width / window level adjustment (250 / 50HU); input clinical parameters including maximum diameter of tumor (unit: cm), anatomical location (head / body / tail of pancreas), patient age and gender. ​

[0044] Lesion segmentation on preprocessed output image I', including tumor region segmentation using improved U-Net model (with position embedding and signed distance field supervision), output binary mask M i and morphological features F v (such as tumor volume, surface area).

[0045] Radiomics features F i are extracted from the segmentation results.

[0046] Texture features: contrast, energy, homogeneity are calculated by gray level co-occurrence matrix (GLCM);

[0047] Shape features: tumor volume, surface area, sphericity are calculated; where sphericity S:

[0048]

[0049] where V is volume, A is surface area.

[0050] Gray level features: mean, variance and kurtosis of CT value histogram are extracted.

[0051] Multi-modal fusion features F m are obtained by cross-modal attention mechanism, which concatenates radiomics features F i (dimension 50) and morphological features F v (dimension 32) into image feature matrix F img (dimension 82), and clinical parameter vector F clin (dimension 5) after standardization. F img and F clin are mapped to query, key, value space by linear transformation.

[0052] Linear transformation is performed on image feature matrix F img and clinical parameters F clin to calculate attention weight matrix A:

[0053]

[0054] where W Q , W k are trainable weight matrices, and d is feature dimension.

[0055] After element-wise multiplication of attention weight and clinical parameter projection value F clin W V and addition with image features, fusion features F fused are obtained:

[0056] F fused = Fimg +A⊙(F clin W V )

[0057] Perform multi-task classification prediction, the multi-task classifier includes:

[0058] Pathology grade prediction head: Input F fused At the fully connected layer, Softmax outputs low-risk / high-risk probabilities. Low-risk includes G1, and high-risk includes G2 and G3. The loss function uses weighted cross-entropy loss to alleviate class imbalance (G1 accounts for 45%, G2 accounts for 50%, and G3 accounts for 5%).

[0059] LNM prediction head: input F fused At the fully connected layer, Sigmoid outputs the risk probability of lymph node metastasis; the loss function uses focal loss (Focal Loss) with parameter γ = 2 to suppress the weight of easily classified samples;

[0060] Joint loss function: cross entropy loss (classification) + focal loss (LNM).

[0061] A dynamic threshold adjustment strategy was adopted to set the initial LNM risk threshold according to preoperative clinical guidelines (such as tumor size ≥ 2 cm), and the threshold was dynamically optimized based on the validation set ROC curve to balance sensitivity and specificity.

[0062] Visualization platform: Outputs interpretable heat maps and labels key areas of model decision-making.

[0063] The visualization platform generates heat maps using class activation mapping (CAM) technology to locate tumor areas that affect classification results and displays them in association with clinical parameters (such as tumor edge irregularity).

[0064] Figure 2 This paper demonstrates the performance evaluation of the proposed multi-task deep learning model in the preoperative prediction of non-functional pancreatic neuroendocrine tumors (NF-PNETs), covering two tasks: tumor pathological grade and lymph node metastasis (LNM) prediction. The specific implementation process includes the following steps:

[0065] First, the model input is the abdominal enhanced CT image (arterial phase and portal phase), which is isotropically resampled (0.7×0.7×1mm 3) and window width / window level adjustment (250 / 50HU) preprocessing. The improved U-Net model (introducing position embedding and signed distance field supervision) was used to realize tumor region segmentation, and morphological features such as volume, surface area, etc. were extracted. Subsequently, radiomics features were extracted from the segmentation mask, including texture features (GLCM contrast, energy, homogeneity), shape features (volume, sphericity) and gray level statistical features (mean, variance, kurtosis). Clinical parameters include tumor maximum diameter, anatomical location (head / body / tail), patient age and gender.

[0066] The model adopts a cross-modal attention mechanism to fuse image features (dimension 82) and clinical parameters (dimension 5), and outputs the pathological grade (Softmax prediction G1 / G2-3) and LNM risk (Sigmoid probability) in parallel through a multi-task structure. The loss functions are weighted cross-entropy and FocalLoss respectively. The final result also combines a dynamic threshold strategy to optimize the classification threshold under the ROC curve based on the reference clinical guidelines (such as tumor size ≥2cm) to balance sensitivity and specificity.

[0067] Figure 2 In the present application, a shows the ROC curve of the tumor grading task on the training set and four independent centers (validation, test, Beijing, Guangzhou), and the AUC value is between 0.704 (Beijing) and 0.763 (validation), indicating that the model has good generalization performance in multi-center data; b is the ROC curve of the LNM prediction task, and the overall AUC performance is better, with a test set AUC of 0.781, and Beijing and Guangzhou centers reaching 0.786 and 0.751, reflecting the strong discriminant ability of the task in clinical significance; c gives the specific diagnostic performance indicators of the tumor grading task. The performance is good on the training and validation sets (accuracy is 0.736 and 0.727 respectively), but the performance decreases on the external test set (accuracy is 0.586), indicating that the model is still limited by the differences between centers; d compares the present model with the traditional clinical LNM prediction method with a threshold of 2cm, and the results show that the present application is significantly better than the traditional empirical method in terms of test set accuracy (0.667 vs 0.596), sensitivity (0.560 vs 0.292) and NPV (0.903 vs 0.739), and has higher clinical application value.

[0068] In summary, the multi-task model proposed in the present application has strong generalization ability and robustness in tumor grading and LNM prediction by fusing radiomics, morphological features and clinical information, and introducing attention mechanism and joint optimization strategy, especially in LNM prediction, which is significantly better than the traditional method, providing an effective auxiliary tool for preoperative accurate evaluation of NF-PNETs.

[0069] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the specific embodiments described above, and various changes or modifications can be made by those skilled in the art within the scope of the claims, which do not affect the essential content of the present application.

Claims

1. A method for predicting preoperative indicators of non-functional pancreatic neuroendocrine tumors is implemented based on medical image processing, characterized in that: The specific steps are: (1) Processing medical images, including preprocessing the input abdominal CT images and performing lesion segmentation on the abdominal CT images based on the improved U-Net model to obtain the segmentation mask and morphological features of the tumor area; (2) Extracting radiomic features from the segmentation mask, including texture features, shape features, and grayscale distribution features; texture features include contrast and energy of the grayscale co-occurrence matrix; shape features include volume and sphericity; and grayscale distribution features include CT value mean and variance; (3) fusing the morphological features, radiomics features, and clinical parameters, including the maximum diameter of the tumor, the location of the pancreatic head / body / tail, and the patient's age; generating multimodal fusion features through a cross-modal attention mechanism; (4) A multi-task classifier was used to jointly predict the pathological grade and lymph node metastasis risk probability, and the classification results were optimized through a dynamic threshold adjustment strategy.

2. The method according to claim 1, characterized in that In step (1): The input data preprocessing includes isotropic resampling and window width / window level adjustment of the input abdominal enhanced CT image; input clinical parameters include maximum tumor diameter, anatomical location, patient age, and gender; The lesion segmentation includes segmenting the tumor area using an improved U-Net model and outputting a binary mask and morphological features, including tumor volume and surface area.

3. The method according to claim 2, characterized in that Extracting radiomic features in step (2) includes: Texture features: contrast, energy, and homogeneity are calculated through gray-level co-occurrence matrix; Shape characteristics: calculation of tumor volume, surface area, and sphericity; Grayscale features: extract the mean, variance and kurtosis of the CT value histogram.

4. The method according to claim 3, characterized in that In step (3), the multimodal fusion features are generated by the cross-modal attention mechanism. Specifically, the imaging omics features and morphological features are spliced ​​into the imaging feature matrix F img , clinical parameter vector F clin Input after standardization; By linear transformation, F img and F clin Map to query, key, and value spaces; Image feature matrix F img and clinical parameters F clin Perform linear transformation and calculate the attention weight matrix A: Among them, W Q ,W K is the trainable weight matrix, d is the feature dimension; The attention weight is combined with the clinical parameter projection value F clin W V After element-by-element multiplication, add it to the image feature to obtain the fusion feature F fused : F fused =F img +A⊙(F clin W V )(2)。 5. The method according to claim 4, characterized in that The multi-task classifier in step (4) includes: Pathology grade prediction head: Input F fused At the fully connected layer, Softmax outputs the low-risk / high-risk probability; low-risk includes G1 period, and high-risk includes G2 and G3 periods; the loss function uses weighted cross entropy loss to alleviate category imbalance; LNM prediction head: input F fused At the fully connected layer, Sigmoid outputs the risk probability of lymph node metastasis; the loss function uses focal loss (FocalLoss) to suppress the weight of easily classified samples; Joint loss function: cross entropy loss (classification) + focal loss (LNM).

6. The method according to claim 5, characterized in that It also includes dynamic threshold adjustment, specifically setting the initial LNM risk threshold according to preoperative clinical guidelines, and dynamically optimizing the threshold based on the validation set ROC curve to balance sensitivity and specificity.

7. A preoperative indicator prediction system for non-functional pancreatic neuroendocrine tumors based on the method according to any one of claims 1 to 6, characterized in that: include: Data preprocessing module, imaging omics feature extraction module, multimodal feature fusion module, multi-task classification module, and visualization platform: Among them, the data preprocessing module, imaging omics feature extraction module, multimodal feature fusion module and multi-task classification module respectively perform the four steps of the prediction method; the visualization platform outputs an interpretable heat map and labels the key decision-making areas of the model. Specifically, the visualization platform generates a heat map through class activation mapping (CAM) technology to locate the tumor area that affects the classification results and displays it in association with clinical parameters.

8. A computer-readable storage medium storing program code for executing a method for predicting preoperative indicators of non-functional pancreatic neuroendocrine tumors.

Images