A method and system for predicting the risk of surgical margin status in cancer

Abstract

The present application relates to the technical field of medical image processing, artificial intelligence assisted surgical planning and computer aided diagnosis and treatment, and provides a cancer surgery margin state risk prediction method and system, the method comprising: acquiring preoperative medical image data of a patient, performing three-dimensional segmentation on a target anatomical structure and a tumor in the medical image data, constructing a three-dimensional anatomical model, and determining a patient-specific planned resection area based on the three-dimensional anatomical model; based on the planned resection area, extracting spatial features from the three-dimensional anatomical model; the spatial features are used to represent the spatial relationship between the tumor and the planned resection plane; extracting multi-scale image features from the medical image data; inputting the spatial features and the multi-scale image features into a multi-modal fusion prediction model to obtain a cancer surgery margin state risk prediction result output by the multi-modal fusion prediction model. The present application can improve the accuracy, stability and clinical interpretability of preoperative margin state risk assessment.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing, artificial intelligence-assisted surgical planning, and computer-aided diagnosis and treatment, and in particular to a method and system for predicting the risk of surgical margin status in cancer surgery. Background Technology

[0002] Surgical margin status is a crucial indicator for assessing the effectiveness of radical cancer resection, directly impacting postoperative survival and recurrence risk. Clinically, R0 resection indicates no tumor residue under a microscope, while R1 resection indicates residual tumor. For biliary malignancies such as hilar cholangiocarcinoma, the infiltration of tumors along the bile duct wall, their hidden growth range, and the complex biliary anatomy make accurate preoperative assessment of the surgical margin status at the planned resection plane extremely difficult. This results in insufficient accuracy in predicting surgical margin risk, hindering the development of individualized surgical plans.

[0003] Current preoperative assessments mainly rely on serum biomarkers, routine image review, empirical 3D visualization, or traditional radiomics models. However, these methods are usually centered on the main tumor and lack joint modeling of patient-specific resection paths, planned resection margin locations, spatial relationships of the biliary tract tree, and background liver status. This makes it difficult to quantitatively predict whether a negative resection margin is likely to be obtained at a specific planned resection plane, and fails to meet the clinical need for precise risk assessment of surgical margin status. Summary of the Invention

[0004] This invention provides a method and system for predicting the risk of surgical margin status in cancer surgery, addressing the shortcomings of existing technologies in meeting the clinical need for accurate assessment of surgical margin status risk. This invention improves the accuracy, stability, and clinical interpretability of preoperative margin status risk assessment.

[0005] This invention provides a method for predicting the risk of surgical margin status in cancer surgery, comprising: acquiring preoperative medical image data of a patient; performing three-dimensional segmentation of the target anatomical structure and tumor in the medical image data; constructing a three-dimensional anatomical model; and determining a patient-specific planned resection region based on the three-dimensional anatomical model; extracting spatial features from the three-dimensional anatomical model based on the planned resection region; the spatial features being used to characterize the spatial relationship between the tumor and the planned resection plane; extracting multi-scale image features from the medical image data; the multi-scale image features including at least features of the target anatomical structure, the tumor, and the planned resection margin region; inputting the spatial features and the multi-scale image features into a multimodal fusion prediction model to obtain the cancer surgical margin status risk prediction result output by the multimodal fusion prediction model; the multimodal fusion prediction model being used to perform feature fusion and classification prediction of the spatial features and the multi-scale image features.

[0006] According to the present invention, a method for predicting the surgical margin status risk of cancer includes three-dimensional segmentation of the target anatomical structure and tumor in the medical image data to construct a three-dimensional anatomical model. This includes: performing three-dimensional segmentation of the biliary tree and main tumor in the patient's preoperative CT image using a preset three-dimensional medical image segmentation model to obtain a biliary tree mask and a main tumor mask; the preset three-dimensional medical image segmentation model is at least one or a combination of 3D nnU-Net, three-dimensional convolutional neural network, three-dimensional Transformer network, and Mamba-type three-dimensional vision network; and constructing a three-dimensional biliary tract-tumor model based on the biliary tree mask and the main tumor mask.

[0007] According to the present invention, a method for predicting the risk of surgical resection margin in cancer surgery, wherein determining the patient-specific planned resection area based on the three-dimensional anatomical model includes: mapping the biliary tree into a bile duct segment to be resected and a bile duct segment to be preserved based on the three-dimensional bile duct-tumor model and a preset surgical plan; and obtaining a patient-specific planned resection plane by dividing the spatial boundary between the bile duct segment to be resected and the bile duct segment to be preserved.

[0008] According to the present invention, a method for predicting the risk of surgical margin status in cancer surgery includes spatial features such as main tumor volume, maximum diameter of main tumor, number of planned bile duct margin sites, minimum distance from planned margin site to the boundary of main tumor, diameter of bile duct at the margin site closest to the main tumor, minimum diameter of margin-related bile duct, maximum diameter of margin-related bile duct, and ratio of planned resected bile duct volume to planned preserved bile duct volume.

[0009] According to the present invention, a method for predicting the risk of surgical margin status in cancer is provided. The step of extracting multi-scale image features from the medical image data includes: extracting local multi-region image features from the medical image data using a preset three-dimensional image encoder; the preset three-dimensional image encoder is a Mamba-type three-dimensional vision network or a three-dimensional convolutional neural network; extracting global image features from the medical image data using a self-supervised learning three-dimensional vision model; the sampling area of ​​the global image features is larger than the sampling area of ​​the local multi-region image features; and the three-dimensional vision model adopts the DINO model.

[0010] According to the present invention, a method for predicting the risk of surgical margin status in cancer surgery includes extracting local multi-region image features from medical image data by means of a preset three-dimensional image encoder. The method comprises: sampling three-dimensional CT local blocks from the main tumor, the tumor-biliary interface and the planned surgical margin area based on the patient's preoperative CT image to form a local block set; inputting the local block set into the preset three-dimensional image encoder to obtain the local multi-region image features output by the three-dimensional image encoder.

[0011] According to the present invention, a method for predicting the risk of surgical margin status in cancer is provided, wherein the global image features include a global image block covering the liver background, tumor context, and peribiliary tissue.

[0012] According to the present invention, a method for predicting the risk of surgical resection margin in cancer surgery includes a multimodal fusion prediction model comprising a spatial mapping module, a fusion module, and a classification module. The step of inputting the spatial features and the multi-scale image features into the multimodal fusion prediction model to obtain the cancer surgical resection margin risk prediction result output by the multimodal fusion prediction model includes: mapping the spatial features, the local multi-region image features, and the global image features to a unified feature space through the spatial mapping module; fusing the features in the unified feature space through the fusion module to obtain fused features; the fusion method includes residual fusion, gated fusion, attention fusion, or multilayer perceptron fusion; and determining, based on the fused features, the positive risk probability, negative risk probability, or R0 / R1 auxiliary classification result corresponding to the patient-specific planned resection plane through the classification module.

[0013] The method for predicting surgical margin status risk according to the present invention further includes: for the same patient, sampling multiple times from the same region to obtain multiple local block sets; based on the multiple local block sets, determining multiple surgical margin status risk prediction results, and averaging the multiple surgical margin status risk prediction results to obtain patient-level surgical margin status risk results.

[0014] The method for predicting the surgical margin status risk of cancer according to the present invention further includes: determining multiple planned resection areas for multiple candidate surgical plans for the same patient; and determining the margin status risk results for each candidate surgical plan based on the multiple planned resection areas, so as to compare or rank the risks of the multiple candidate surgical plans.

[0015] This invention also provides a cancer surgical margin status risk prediction system, comprising: a region determination module, used to acquire preoperative medical image data of a patient, perform three-dimensional segmentation of the target anatomical structure and tumor in the medical image data, construct a three-dimensional anatomical model, and determine the patient-specific planned resection region based on the three-dimensional anatomical model; a spatial feature extraction module, used to extract spatial features from the three-dimensional anatomical model based on the planned resection region; the spatial features are used to characterize the spatial relationship between the tumor and the planned resection plane; a multi-scale image feature extraction module, used to extract multi-scale image features from the medical image data; the multi-scale image features include at least features of the target anatomical structure, the tumor, and the planned resection margin region; and a prediction result determination module, used to input the spatial features and the multi-scale image features into a multimodal fusion prediction model to obtain the cancer surgical margin status risk prediction result output by the multimodal fusion prediction model; the multimodal fusion prediction model is used to perform feature fusion and classification prediction of the spatial features and the multi-scale image features.

[0016] According to the present invention, a cancer surgical margin risk prediction system includes a region determination module comprising: a model construction module, used to perform three-dimensional segmentation of the biliary tree and the main tumor in the patient's preoperative CT image using a preset three-dimensional medical image segmentation model to obtain a biliary tree mask and a main tumor mask; the preset three-dimensional medical image segmentation model is at least one or a combination of 3D nnU-Net, three-dimensional convolutional neural network, three-dimensional Transformer network, and Mamba-type three-dimensional vision network; and a three-dimensional biliary tract-tumor model is constructed based on the biliary tree mask and the main tumor mask.

[0017] According to the cancer surgical margin status risk prediction system provided by the present invention, the region determination module further includes: a planned resection plane determination module, used to map the bile duct tree into a bile duct segment to be resected and a bile duct segment to be preserved based on the three-dimensional bile duct-tumor model and the preset surgical plan; and to obtain a patient-specific planned resection plane by dividing the spatial boundary between the bile duct segment to be resected and the bile duct segment to be preserved.

[0018] According to the present invention, a risk prediction system for surgical margin status in cancer surgery includes spatial features such as main tumor volume, maximum diameter of main tumor, number of planned bile duct margin sites, minimum distance from planned margin site to main tumor boundary, bile duct diameter at the margin site closest to the main tumor, minimum diameter of margin-related bile duct, maximum diameter of margin-related bile duct, and ratio of planned resected bile duct volume to planned preserved bile duct volume.

[0019] According to the present invention, a cancer surgical margin risk prediction system includes a multi-scale image feature extraction module comprising: a local feature extraction module for extracting local multi-region image features from the medical image data using a preset three-dimensional image encoder; the preset three-dimensional image encoder is a Mamba-type three-dimensional vision network or a three-dimensional convolutional neural network; and a global feature extraction module for extracting global image features from the medical image data using a self-supervised learning three-dimensional vision model; the sampling area of ​​the global image features is larger than the sampling area of ​​the local multi-region image features; and the three-dimensional vision model adopts the DINO model.

[0020] According to the present invention, a cancer surgical margin status risk prediction system is provided, wherein the multimodal fusion prediction model includes a spatial mapping module, a fusion module, and a classification module; the prediction result determination module is used to: map the spatial features, the local multi-region image features, and the global image features to a unified feature space through the spatial mapping module; fuse the features of the unified feature space through the fusion module to obtain fused features; the fusion method includes residual fusion, gated fusion, attention fusion, or multilayer perceptron fusion; and determine the positive risk probability, negative margin probability, or R0 / R1 auxiliary classification result corresponding to the patient-specific planned resection plane based on the fused features through the classification module.

[0021] The present invention provides a method and system for predicting the risk of surgical margin status in cancer surgery, which can improve the accuracy, stability and clinical interpretability of preoperative surgical margin status risk assessment. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the cancer surgical margin status risk prediction method provided by the present invention.

[0024] Figure 2 This is a schematic diagram of the cancer surgical margin status risk prediction system provided by the present invention.

[0025] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating the method for predicting the risk of surgical margin status in cancer surgery provided by the present invention.

[0028] This invention provides a method for predicting the risk of surgical margin status in cancer surgery, comprising: 101: Acquire the patient's preoperative medical image data, perform three-dimensional segmentation of the target anatomical structures and tumors in the medical image data, construct a three-dimensional anatomical model, and determine the patient-specific planned resection area based on the three-dimensional anatomical model; 102: Based on the planned resection area, spatial features are extracted from the three-dimensional anatomical model; the spatial features are used to characterize the spatial relationship between the tumor and the planned resection plane; 103: Extract multi-scale image features from medical image data; the multi-scale image features should include at least the features of the target anatomical structure, tumor, and planned resection margin region; 104: Input spatial features and multi-scale image features into the multimodal fusion prediction model to obtain the cancer surgical margin status risk prediction results output by the multimodal fusion prediction model; the multimodal fusion prediction model is used to perform feature fusion and classification prediction of spatial features and multi-scale image features.

[0029] This invention provides a method for predicting the risk of surgical margin status in cancer, using hilar cholangiocarcinoma as an example. Preoperative medical image data of the patient is acquired, which can be at least one of CT images, MRI images, and ultrasound images. The target anatomical structure and tumor in the medical image data are segmented in three dimensions to construct a three-dimensional anatomical model; the target anatomical structure can be the biliary tree in the CT image, and the corresponding three-dimensional anatomical model is a three-dimensional biliary-tumor model. Based on the three-dimensional anatomical model, the patient-specific planned resection area is determined; the planned resection area can be the planned resection plane, and the three-dimensional anatomical model can accurately depict the biliary anatomy, tumor extent, and the spatial relationship between the lesion and the planned resection plane. Based on the planned resection area and the three-dimensional anatomical model, spatial features characterizing the spatial relationship between the tumor and the planned resection plane are extracted. Furthermore, multi-scale image features are extracted from the medical image data, which can integrate multi-regional image features of the main tumor, the tumor-biliary interface, resection margin features, and global background liver features. Through a multi-modal fusion prediction model, feature fusion and classification prediction are performed on the spatial features and multi-scale image features, outputting the predicted risk of surgical margin status in cancer. The Multimodal Adaptive Resection-Guided Integration Network (MARGIN) of this invention, based on a CT framework, integrates spatial features, multi-regional image features of the main tumor, tumor-biliary interface and resection margin features, as well as global background liver features. By combining patient-specific resection plane reconstruction with multi-scale CT characterization, it achieves quantitative and individualized prediction of resection margin status, thereby supporting individualized radical surgical planning.

[0030] As a preferred embodiment, the target anatomical structure and tumor in medical image data are segmented in three dimensions to construct a three-dimensional anatomical model, including: performing three-dimensional segmentation of the biliary tree and the main tumor in the patient's preoperative CT image using a preset three-dimensional medical image segmentation model to obtain a biliary tree mask and a main tumor mask; the preset three-dimensional medical image segmentation model is at least one or a combination of 3D nnU-Net, three-dimensional convolutional neural network, three-dimensional Transformer network, and Mamba-type three-dimensional vision network; and constructing a three-dimensional biliary tract-tumor model based on the biliary tree mask and the main tumor mask.

[0031] In this embodiment, the patient's preoperative enhanced CT images, such as CT image data of the biliary tree, tumor, and liver, can be resampled, intensity normalized, spatially registered, or cropped. A three-dimensional medical image segmentation model is used to automatically segment the biliary tree and main tumor in the preoperative CT images. The segmentation results can be corrected by professionals to obtain biliary tree masks and main tumor masks for modeling. A three-dimensional biliary-tumor model is reconstructed based on the biliary tree mask and main tumor mask, improving the practical decision support value of three-dimensional modeling in preoperative planning.

[0032] The 3D medical image segmentation model was trained based on 197 manually annotated CT scans of hilar cholangiocarcinoma. For example, the trained 3D nnU-Net can automatically segment the biliary tree and the main tumor. The segmentation results can be reviewed and optimized by two senior radiologists. The CT images and corresponding segmentation masks were resampled to an isotropic voxel spacing of 1mm×1mm×1mm to reduce the impact of different scanning equipment, different centers, and different slice thicknesses on feature calculation.

[0033] 3D nnU-Net was used as the segmentation network to perform end-to-end 3D semantic segmentation on preoperative enhanced CT images of patients to obtain biliary tree masks and master tumor masks. 3D nnU-Net is a self-configuring deep learning method that automatically adapts the network topology based on attributes such as voxel spacing and image size of the input data, without requiring manual adjustment of network hyperparameters. 3D nnU-Net employs an encoder-decoder architecture. The encoder consists of multiple cascaded downsampling blocks, each containing two 3×3×3 3D convolutional layers. Each convolutional layer is followed by instance normalization and a Leaky ReLU activation function, and then spatial resolution downsampling is achieved through 3×3×3 convolutions with a stride of 2, progressively extracting multi-scale feature representations from low-level edge features to high-level semantic features. The decoder consists of multiple cascaded upsampling blocks. Each upsampling block restores the spatial resolution of the feature map through transposed convolution or upsampling concatenation operations and performs skip connections with the feature maps of the corresponding layers in the encoder to fuse high-resolution detail features with deep semantic features, mitigating spatial information loss during downsampling. 3DnnU-Net employs a U-shaped symmetrical structure, with the encoder and decoder connected via a bottleneck layer. The encoder path progressively reduces spatial resolution and increases the number of feature channels layer by layer, while the decoder path progressively restores spatial resolution and reduces the number of feature channels layer by layer. Skip connections directly concatenate feature maps of the same resolution from the encoder to the corresponding levels in the decoder, allowing the network to preserve shallow spatial details while extracting deep features, thus improving the segmentation accuracy for small tubular structures such as bile duct trees. Parameters such as network depth, number of convolutional kernels, and input image patch size are automatically inferred from image attributes in the training dataset (e.g., median voxel spacing and image size). For bile duct tree and main tumor segmentation tasks, the input is 3D CT volumetric data, and the output is a probability map of the same size as the input. After Softmax activation, the class with the highest probability is selected to generate binary segmentation results for the bile duct tree mask and the main tumor mask.

[0034] As a preferred embodiment, determining the patient-specific planned resection area based on a three-dimensional anatomical model includes: mapping the bile duct tree into the bile duct segment to be resected and the bile duct segment to be preserved based on a three-dimensional bile duct-tumor model and a pre-set surgical plan; and obtaining the patient-specific planned resection plane by defining the spatial boundary between the bile duct segment to be resected and the bile duct segment to be preserved.

[0035] In this embodiment, based on a three-dimensional bile duct-tumor model and surgical records, preoperative planning information, physician interactive input, or candidate surgical plans, the bile duct tree is mapped to the bile duct segment to be resected and the bile duct segment to be preserved. The spatial boundary between the bile duct segment to be resected and the bile duct segment to be preserved serves as a patient-specific planning resection plane, shifting the prediction focus from the general patient or main tumor to the specific surgical path and resection margin area, which better meets the needs of radical surgical planning for hilar cholangiocarcinoma.

[0036] The planned resection plane can correspond to the actual surgical path of left hemihepatectomy, right hemihepatectomy, left trihepatectomy, right trihepatectomy, or other radical surgical approaches for hilar cholangiocarcinoma, or it can correspond to the candidate resection plans constructed by the surgeon during the preoperative planning stage. For the same patient, the system can reconstruct multiple candidate resection planes and calculate the risk output for each.

[0037] The method for determining the planned resection plane is not limited to postoperative retrospective reconstruction, but can also come from preoperative physician interactive planning, regular surgical template matching, candidate resection path search, or automatically generated multi-plan resection simulation.

[0038] As a preferred embodiment, the spatial features include the volume of the main tumor, the maximum diameter of the main tumor, the number of planned bile duct resection sites, the minimum distance from the planned resection site to the boundary of the main tumor, the diameter of the bile duct at the resection site closest to the main tumor, the minimum diameter of the bile duct related to the resection site, the maximum diameter of the bile duct related to the resection site, and the ratio of the volume of the bile duct to be resected to the volume of the bile duct to be preserved.

[0039] In this embodiment, eight spatial features are extracted based on the three-dimensional bile duct-tumor model and the planned resection plane to characterize the resection plane-specific anatomical and tumor-related information, including tumor burden, planned bile duct resection margin features, and the spatial relationship between the tumor and bile duct. The spatial features can be standardized using normalized parameters fitted to the training set, and the same normalization rule is maintained during the validation or inference phases to ensure a consistent model input distribution.

[0040] Spatial features are not limited to eight indicators; additional indicators can be added, such as bile duct bifurcation level, normal distance of the cutting edge, tumor encirclement angle, bile duct skeleton topological indicators, local curvature, vascular distance, or liver segment volume information.

[0041] As a preferred embodiment, extracting multi-scale image features from medical image data includes: extracting local multi-region image features from medical image data using a preset 3D image encoder; the preset 3D image encoder is a Mamba-type 3D vision network or a 3D convolutional neural network; extracting global image features from medical image data using a self-supervised learning 3D vision model; the sampling area of ​​the global image features is larger than the sampling area of ​​the local multi-region image features; and the 3D vision model adopts the DINO model.

[0042] As a preferred embodiment, local multi-region image features are extracted from medical image data by means of a preset three-dimensional image encoder, including: sampling three-dimensional CT local blocks from the main tumor, the tumor-biliary interface and the planned resection margin area based on the patient's preoperative CT image to form a local block set; inputting the local block set into the preset three-dimensional image encoder to obtain the local multi-region image features output by the three-dimensional image encoder.

[0043] As a preferred embodiment, the method further includes: for the same patient, multiple samplings are performed from the same region to obtain multiple local block sets; based on the multiple local block sets, multiple resection margin risk prediction results are determined, and the multiple resection margin risk prediction results are averaged to obtain patient-level resection margin risk results.

[0044] To capture local image heterogeneity associated with the risk of positive surgical margins, this embodiment defines at least three local regions in the 3D CT image: the main tumor region, the tumor-biliary interface region, and the planned surgical margin region. One or more 3D local image blocks are sampled from each region, and these local blocks are grouped into a local block set. For example, two 32×32×32 voxel 3D CT local blocks are sampled from each of the main tumor, the tumor-biliary interface, and the planned surgical margin region, resulting in a total of six local blocks forming a local block set. The six local blocks are input into a 3D image encoder with shared weights, such as a 3D Mamba or 3D convolutional network. The local block features are then subjected to adaptive average pooling, stitching, and linear mapping to form a unified local image feature vector.

[0045] For the same patient, multiple samples can be taken from the same region to form multiple local block sets. During the training phase, multiple local block sets are treated as set-level samples to increase the diversity of local sampling; during the inference phase, the multiple set-level prediction results for the same patient are averaged or weighted to form a patient-level prediction result.

[0046] The local region is not limited to three regions and six local blocks. The number of local blocks, scale, and sampling strategy can be adaptively adjusted according to tumor size, biliary tree topology, and number of candidate resection margins.

[0047] Local image encoders are not limited to 3D Mamba; they can use 3D ResNet, DenseNet, SwinTransformer, ViT, ConvNeXt, state-space models, or multi-scale encoders.

[0048] A 3D Mamba network is used as the backbone network for local image feature encoding to extract features from multi-region 3D image patches sampled from medical image data. 3D Mamba is an efficient sequence modeling method for 3D medical data. It achieves long-distance dependency modeling with linear time complexity through a selective state-space model, overcoming the bottlenecks of high computational complexity and large memory consumption of traditional Transformers in 3D medical image processing.

[0049] The 3D Mamba algorithm employs a hierarchical coding architecture. The input layer divides 3D image blocks (e.g., 32×32×32 voxels) into 3D non-overlapping image patches. Each patch is embedded into a feature vector via 3D linear projection, and 3D positional encoding is superimposed to preserve spatial location information. The coding layer consists of multiple stacked 3D Mamba blocks. Each 3D Mamba block contains a normalization layer, a 3D selective scan mechanism, and a feedforward network. The 3D selective scan mechanism performs selective state-space transformations along three orthogonal directions (X-axis, Y-axis, Z-axis), selectively focusing on the input sequence through learnable parameters, dynamically adjusting the state transition matrix and projection matrix. This allows the network to retain global context awareness while focusing on local pathological features related to the cutting edge state. Within each 3D Mamba block, an improved state-space equation describes the sequence dynamics: by discretizing the continuous state-space model, the input sequence is mapped to the hidden state space and then projected to the output space. A selective mechanism introduces an input-dependent gating strategy, enabling the state-space parameters to adaptively adjust with the input content, achieving selective memorization and transmission of key regions such as tumor boundaries and biliary invasion in 3D medical images. Scanning results from three orthogonal directions are aggregated through a fusion layer to form a feature representation containing long-distance correlations in 3D space. Multiple 3D local image blocks are sampled for the main tumor region, the tumor-anatomical interface region, and the planned resection margin region. Each image block is independently encoded by a 3D Mamba backbone network with shared weights, and the block-level feature map is compressed into a fixed-dimensional feature vector through adaptive average pooling. The feature vectors of multiple regions from the same patient are concatenated in a preset order, and after dimensionality reduction via linear projection, a unified local multi-region image feature representation is generated for subsequent multimodal fusion and resection margin risk prediction.

[0050] As a preferred embodiment, the global image features include global image patches covering the liver background, tumor context, and peribiliary tissue.

[0051] To supplement local multi-region image features, this embodiment also extracts global image representations from a larger CT image patch. This global image patch preferably covers the liver background, tumor context, peribiliary tissue, and image patterns potentially related to prognosis or resection margin risk. For example, the global CT image patch size is 112×112×112 voxels. Global features are extracted using a 3D vision foundation model that is continuously pre-trained based on a 3D self-supervised learning framework, such as DINO, using the AbdomenAtlas dataset (containing over 9,000 abdominal CT scans). Candidate global features are derived from the outputs of the last four Transformer blocks and are combined with class label vectors and image patch label statistical vectors to simultaneously express global semantic information and distributed texture patterns.

[0052] To reduce redundancy and improve generalization, low variance filtering, univariate screening, collinearity reduction, Bootstrap stability analysis, and LASSO selection can be performed on the training set to retain stable global image features as input for downstream multimodal fusion.

[0053] The 3D vision base model is not limited to a specific DINO implementation; any 3D CT base model pre-trained based on self-supervised, weakly supervised, or supervised learning can be used.

[0054] A 3D vision foundation model is constructed using the DINO self-supervised pre-training framework to extract global image features from medical image data. DINO is a self-distillation learning framework for 3D medical images. Through a teacher-student network structure, it achieves feature representation learning without manual annotation, capturing global semantic information and distributed texture patterns to represent background organ states and tumor-organ contextual relationships. DINO employs a dual-branch asymmetric encoder architecture. Both the teacher and student branches consist of a 3D vision Transformer (3D ViT). The teacher branch receives global image patches (e.g., 112×112×112 voxels) as input, while the student branch receives masked global and local image patches as input.

[0055] As a preferred embodiment, the multimodal fusion prediction model includes a spatial mapping module, a fusion module, and a classification module. Spatial features and multi-scale image features are input into the multimodal fusion prediction model to obtain the cancer surgical margin risk prediction result output by the multimodal fusion prediction model. This includes: mapping spatial features, local multi-region image features, and global image features to a unified feature space through the spatial mapping module; fusing the features in the unified feature space through the fusion module to obtain fused features; the fusion method includes residual fusion, gated fusion, attention fusion, or multilayer perceptron fusion; and determining the positive risk probability, negative margin probability, or R0 / R1 auxiliary classification result of the planned resection plane specific to the corresponding patient through the classification module.

[0056] In this embodiment, spatial features and multi-scale image features are input into a multimodal fusion prediction model to obtain the cancer surgical margin risk prediction result output by the multimodal fusion prediction model. Specifically, spatial feature vectors, local image feature vectors, and global image feature vectors are mapped to a unified feature space and integrated using residual fusion, gated fusion, attention fusion, or multilayer perceptron fusion methods. The fused features are input into a classifier or risk regressor, which outputs the positive risk probability, negative margin probability, or R0 / R1 auxiliary classification result corresponding to the preset resection plane.

[0057] The multimodal fusion prediction model employs a residual multimodal fusion structure, ensuring that each feature retains its own expression while complementing other feature flows. It takes into account anatomical structures, the tumor-biliary interface, the planned resection margin region, and the background liver condition, improving external validation performance and generalization ability. This invention can simultaneously utilize spatial relationships related to the resection plane, local tumor-biliary-resection margin image information, and global liver background characterization, reducing the sensitivity of a single feature source to changes in external data distribution.

[0058] Multimodal fusion is not limited to residual fusion; it can employ early fusion, late fusion, cross-attention, gated expert hybrid, graph neural network, or probabilistic calibration model.

[0059] The output is not limited to binary classification and can be extended to risk probability, continuous risk score, multi-level risk stratification, resection margin area localization map or ranking of multiple candidate surgical plans.

[0060] As a preferred embodiment, the method further includes: for multiple candidate surgical options for the same patient, determining multiple planned resection areas; and based on the multiple planned resection areas, determining the risk outcome of the resection margin status for each candidate surgical option, so as to compare or rank the risks of the multiple candidate surgical options.

[0061] This invention mitigates local sampling uncertainty and improves the stability of patient-level output by employing aggregate-level sampling and patient-level aggregation. Through risk probability and multi-option comparison outputs, it provides objective auxiliary information for physicians to assess whether to adjust the resection plane, expand the resection range, or reassess the surgical benefit.

[0062] The cancer surgical margin risk prediction results output by this invention include, but are not limited to: margin risk probability of the planned resection plane, R0 / R1 auxiliary classification results, risk threshold prompts, spatial positioning information of the corresponding margin area, risk comparison results of different candidate resection schemes, and three-dimensional bile duct-tumor-resection plane visualization results for physician review.

[0063] If the predicted risk of R1 at a planned resection level is high, the physician can further assess whether to expand the resection area or consider other treatment strategies, taking into account the patient's liver reserve, vascular invasion, distant metastasis, biliary anatomy, and other clinical factors. The system does not directly make a treatment decision but provides auxiliary calculations based on image and structural information.

[0064] This study included 410 patients with pathologically confirmed hilar cholangiocarcinoma who underwent radical resection with extensive hepatectomy and caudate lobe resection. Data from the development cohort of 234 patients were used for model training and internal validation, while data from the external validation cohort of 176 patients were used for external validation. The system used preoperative enhanced CT to segment the biliary tree and main tumor, completed three-dimensional biliary tract modeling and surgical path reconstruction, and extracted spatial, local, and global features for the actual planned resection plane.

[0065] For the external validation queue, the MARGIN (Multimodal Adaptive Resection Guided Integration Network) model, which integrates spatial, local, and global features, achieved better overall performance, with an AUROC of 0.87, an accuracy of 0.84, a sensitivity of 0.76, a specificity of 0.87, a positive prediction value of 0.72, a negative prediction value of 0.89, and an F1 score of 0.74. The multimodal fusion scheme demonstrated good overall generalization performance.

[0066] Please refer to Table 1. Using physician-assisted assessment, five physicians evaluated the surgical margin status of the external validation cohort under both system-assisted and non-system-assisted conditions. After system assistance, physician accuracy improved from 0.69-0.74 to 0.83-0.85, sensitivity improved from 0.15-0.30 to 0.56-0.67, positive predictive value improved from 0.50-0.67 to 0.79-0.84, and F1 score improved from 0.24-0.41 to 0.67-0.74.

[0067] Patients predicted by the system to have an R1 risk had significantly worse overall survival and recurrence-free survival than patients predicted to have an R0 risk. This indicates that the risk output not only corresponds to the pathological resection margin status but also has a consistent correlation with postoperative outcomes. This invention can improve the accuracy of preoperative resection margin risk prediction for hilar cholangiocarcinoma and provide clinically interpretable auxiliary planning information.

[0068] Table 1 Comparison of Edge Risk Prediction Performance Here, the P-value represents a paired comparison between each surgeon's assessment without systematic assistance and assessment with systematic assistance. Abbreviations: -AI, assessment without systematic assistance; +AI, assessment with systematic assistance.

[0069] The cancer surgical margin risk prediction system provided by the present invention is described below. The cancer surgical margin risk prediction system described below can be referred to in correspondence with the cancer surgical margin risk prediction method described above.

[0070] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the cancer surgical margin status risk prediction system provided by the present invention.

[0071] This invention also provides a cancer surgical margin status risk prediction system, comprising: a region determination module 201, used to acquire preoperative medical image data of the patient, perform three-dimensional segmentation of the target anatomical structure and tumor in the medical image data, construct a three-dimensional anatomical model, and determine the patient-specific planned resection area based on the three-dimensional anatomical model; a spatial feature extraction module 202, used to extract spatial features from the three-dimensional anatomical model based on the planned resection area; the spatial features are used to characterize the spatial relationship between the tumor and the planned resection plane; a multi-scale image feature extraction module 203, used to extract multi-scale image features from the medical image data; the multi-scale image features include at least the features of the target anatomical structure, the tumor, and the planned resection margin area; and a prediction result determination module 204, used to input the spatial features and multi-scale image features into a multi-modal fusion prediction model to obtain the cancer surgical margin status risk prediction result output by the multi-modal fusion prediction model; the multi-modal fusion prediction model is used to perform feature fusion and classification prediction of the spatial features and multi-scale image features.

[0072] In a preferred embodiment, the region determination module includes: a model building module, used to perform three-dimensional segmentation of the biliary tree and the main tumor in the patient's preoperative CT image using a preset three-dimensional medical image segmentation model to obtain a biliary tree mask and a main tumor mask; the preset three-dimensional medical image segmentation model is at least one or a combination of 3D nnU-Net, three-dimensional convolutional neural network, three-dimensional Transformer network, and Mamba-type three-dimensional vision network; and a three-dimensional biliary tract-tumor model is constructed based on the biliary tree mask and the main tumor mask.

[0073] In a preferred embodiment, the region determination module further includes: a planned resection plane determination module, used to map the bile duct tree into bile duct segments to be resected and bile duct segments to be preserved based on a three-dimensional bile duct-tumor model and a preset surgical plan; and to obtain a patient-specific planned resection plane by defining the spatial boundary between the bile duct segments to be resected and the bile duct segments to be preserved.

[0074] As a preferred embodiment, the spatial features include the volume of the main tumor, the maximum diameter of the main tumor, the number of planned bile duct resection sites, the minimum distance from the planned resection site to the boundary of the main tumor, the diameter of the bile duct at the resection site closest to the main tumor, the minimum diameter of the bile duct related to the resection site, the maximum diameter of the bile duct related to the resection site, and the ratio of the volume of the bile duct to be resected to the volume of the bile duct to be preserved.

[0075] In a preferred embodiment, the multi-scale image feature extraction module includes: a local feature extraction module, used to extract local multi-region image features from medical image data through a preset 3D image encoder; the preset 3D image encoder is a Mamba-type 3D vision network or a 3D convolutional neural network; and a global feature extraction module, used to extract global image features from medical image data through a self-supervised learning 3D vision model; the sampling area of ​​the global image features is larger than the sampling area of ​​the local multi-region image features; and the 3D vision model adopts the DINO model.

[0076] As a preferred embodiment, the multimodal fusion prediction model includes a spatial mapping module, a fusion module, and a classification module. The prediction result determination module is used to: map spatial features, local multi-region image features, and global image features to a unified feature space through the spatial mapping module; fuse the features in the unified feature space through the fusion module to obtain fused features; the fusion method includes residual fusion, gated fusion, attention fusion, or multilayer perceptron fusion; and determine the positive risk probability, negative margin probability, or R0 / R1 auxiliary classification result of the planned resection plane specific to the corresponding patient based on the fused features through the classification module.

[0077] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include: a processor 301, a communications interface 302, a memory 303, and a communications bus 304, wherein the processor 301, the communications interface 302, and the memory 303 communicate with each other through the communications bus 304. The processor 301 can call logical instructions in the memory 303 to execute a cancer surgical margin risk prediction method. This method includes: acquiring preoperative medical image data of the patient; performing three-dimensional segmentation of the target anatomical structure and tumor in the medical image data; constructing a three-dimensional anatomical model; and determining the patient-specific planned resection area based on the three-dimensional anatomical model; extracting spatial features from the three-dimensional anatomical model based on the planned resection area; the spatial features are used to characterize the spatial relationship between the tumor and the planned resection plane; extracting multi-scale image features from the medical image data; the multi-scale image features include at least features of the target anatomical structure, tumor, and planned resection margin area; inputting the spatial features and multi-scale image features into a multimodal fusion prediction model to obtain the cancer surgical margin risk prediction result output by the multimodal fusion prediction model; the multimodal fusion prediction model is used to perform feature fusion and classification prediction of the spatial features and multi-scale image features.

[0078] Furthermore, the logical instructions in the aforementioned memory 303 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0079] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the cancer surgical margin risk prediction method provided by the above methods. The method includes: acquiring preoperative medical image data of the patient; performing three-dimensional segmentation of the target anatomical structure and tumor in the medical image data; constructing a three-dimensional anatomical model; and determining the patient-specific planned resection area based on the three-dimensional anatomical model; extracting spatial features from the three-dimensional anatomical model based on the planned resection area; the spatial features are used to characterize the spatial relationship between the tumor and the planned resection plane; extracting multi-scale image features from the medical image data; the multi-scale image features include at least the features of the target anatomical structure, the tumor, and the planned resection margin area; inputting the spatial features and multi-scale image features into a multimodal fusion prediction model to obtain the cancer surgical margin risk prediction result output by the multimodal fusion prediction model; the multimodal fusion prediction model is used to perform feature fusion and classification prediction of the spatial features and multi-scale image features.

[0080] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for predicting the risk of surgical margin status in cancer provided by the methods described above. This method includes: acquiring preoperative medical image data of a patient; performing three-dimensional segmentation of the target anatomical structure and tumor in the medical image data; constructing a three-dimensional anatomical model; and determining a patient-specific planned resection region based on the three-dimensional anatomical model; extracting spatial features from the three-dimensional anatomical model based on the planned resection region; the spatial features being used to characterize the spatial relationship between the tumor and the planned resection plane; extracting multi-scale image features from the medical image data; the multi-scale image features including at least features of the target anatomical structure, the tumor, and the planned resection margin region; inputting the spatial features and multi-scale image features into a multimodal fusion prediction model to obtain a cancer surgical margin status risk prediction result output by the multimodal fusion prediction model; the multimodal fusion prediction model being used to perform feature fusion and classification prediction of the spatial features and multi-scale image features.

[0081] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting the risk of surgical margin status in cancer surgery, characterized in that, include: Acquire preoperative medical image data of the patient, perform three-dimensional segmentation of the target anatomical structures and tumors in the medical image data, construct a three-dimensional anatomical model, and determine the patient-specific planned resection area based on the three-dimensional anatomical model; Based on the planned resection area, spatial features are extracted from the three-dimensional anatomical model; The spatial features are used to characterize the spatial relationship between the tumor and the planned resection plane; Multi-scale image features are extracted from the medical image data; the multi-scale image features include at least the features of the target anatomical structure, tumor, and planned resection margin region. The spatial features and the multi-scale image features are input into the multimodal fusion prediction model to obtain the cancer surgical margin risk prediction result output by the multimodal fusion prediction model; the multimodal fusion prediction model is used to perform feature fusion and classification prediction on the spatial features and the multi-scale image features.

2. The method for predicting the risk of surgical margin status in cancer according to claim 1, characterized in that, The step of performing three-dimensional segmentation of the target anatomical structures and tumors in the medical image data and constructing a three-dimensional anatomical model includes: Using a pre-defined three-dimensional medical image segmentation model, the biliary tree and main tumor in the patient's preoperative CT image are segmented in three dimensions to obtain the biliary tree mask and the main tumor mask; the pre-defined three-dimensional medical image segmentation model is at least one or a combination of 3D nnU-Net, three-dimensional convolutional neural network, three-dimensional Transformer network, and Mamba-type three-dimensional vision network. Based on the biliary tree mask and the main tumor mask, a three-dimensional biliary tract-tumor model is constructed.

3. The method for predicting the risk of surgical margin status in cancer according to claim 2, characterized in that, The process of determining the patient-specific planned resection area based on the three-dimensional anatomical model includes: Based on the aforementioned three-dimensional biliary tract-tumor model and the pre-designed surgical plan, the biliary tract tree is mapped to the biliary tract segments to be resected and the biliary tract segments to be preserved. The spatial boundary between the bile duct segment to be resected and the bile duct segment to be preserved is used to obtain a patient-specific planned resection plane.

4. The method for predicting the risk of surgical margin status in cancer according to claim 2, characterized in that, The spatial characteristics include the volume of the main tumor, the maximum diameter of the main tumor, the number of planned bile duct resection sites, the minimum distance from the planned resection site to the boundary of the main tumor, the diameter of the bile duct at the resection site closest to the main tumor, the minimum diameter of the bile duct related to the resection site, the maximum diameter of the bile duct related to the resection site, and the ratio of the volume of the bile duct to be resected to the volume of the bile duct to be preserved.

5. The method for predicting the risk of surgical margin status in cancer according to claim 2, characterized in that, The extraction of multi-scale image features from the medical image data includes: Local multi-region image features are extracted from the medical image data by a preset 3D image encoder; the preset 3D image encoder is a Mamba-type 3D vision network or a 3D convolutional neural network. A self-supervised learning 3D vision model extracts global image features from the medical image data; the sampling area of ​​the global image features is larger than the sampling area of ​​the local multi-region image features; the 3D vision model adopts the DINO model.

6. The method for predicting the risk of surgical margin status in cancer according to claim 5, characterized in that, The step of extracting local multi-region image features from the medical image data using a preset three-dimensional image encoder includes: Based on the patient’s preoperative CT images, three-dimensional CT local blocks were sampled from the main tumor, the tumor-biliary interface and the planned resection margin area to form a local block set; The local block set is input into a preset three-dimensional image encoder to obtain the local multi-region image features output by the three-dimensional image encoder.

7. The method for predicting the risk of surgical margin status in cancer according to claim 5, characterized in that, The global image features include global image patches covering the liver background, tumor context, and peribiliary tissue.

8. The method for predicting the risk of surgical margin status in cancer according to claim 5, characterized in that, The multimodal fusion prediction model includes a spatial mapping module, a fusion module, and a classification module; The step of inputting the spatial features and the multi-scale image features into a multimodal fusion prediction model to obtain the cancer surgical margin risk prediction result output by the multimodal fusion prediction model includes: The spatial mapping module maps the spatial features, the local multi-region image features, and the global image features to a unified feature space. The fusion module fuses features in a unified feature space to obtain fused features; the fusion methods include residual fusion, gating fusion, attention fusion, or multilayer perceptron fusion. The classification module determines the probability of positive or negative margins, or the R0 / R1 auxiliary classification result for the planned resection plane specific to the patient, based on the fused features.

9. The method for predicting the risk of surgical margin status in cancer according to claim 6, characterized in that, Also includes: For the same patient, multiple samples are taken from the same area to obtain multiple sets of local blocks; Based on multiple local block sets, multiple resection margin status risk prediction results are determined, and the average of these multiple resection margin status risk prediction results is processed to obtain patient-level resection margin status risk results.

10. The method for predicting the risk of surgical margin status in cancer according to any one of claims 1 to 9, characterized in that, Also includes: For multiple candidate surgical plans for the same patient, identify multiple planned resection areas; Based on multiple planned resection areas, the risk outcome of the resection margin status for each candidate surgical approach is determined in order to compare or rank the risks of multiple candidate surgical approaches.

11. A risk prediction system for surgical margin status in cancer, characterized in that, include: The region determination module is used to acquire the patient's preoperative medical image data, perform three-dimensional segmentation of the target anatomical structures and tumors in the medical image data, construct a three-dimensional anatomical model, and determine the patient-specific planned resection area based on the three-dimensional anatomical model. A spatial feature extraction module is used to extract spatial features from the three-dimensional anatomical model based on the planned resection area; The spatial features are used to characterize the spatial relationship between the tumor and the planned resection plane; A multi-scale image feature extraction module is used to extract multi-scale image features from the medical image data; the multi-scale image features include at least features of the target anatomical structure, tumor, and planned resection margin region. The prediction result determination module is used to input the spatial features and the multi-scale image features into the multimodal fusion prediction model to obtain the cancer surgical margin status risk prediction result output by the multimodal fusion prediction model; the multimodal fusion prediction model is used to perform feature fusion and classification prediction on the spatial features and the multi-scale image features.

12. The cancer surgical margin status risk prediction system according to claim 11, characterized in that, The region determination module includes: The model building module is used to perform three-dimensional segmentation of the biliary tree and the main tumor in the patient's preoperative CT image using a preset three-dimensional medical image segmentation model, to obtain a biliary tree mask and a main tumor mask; the preset three-dimensional medical image segmentation model is at least one or a combination of 3D nnU-Net, three-dimensional convolutional neural network, three-dimensional Transformer network, and Mamba-type three-dimensional vision network; and a three-dimensional biliary tract-tumor model is constructed based on the biliary tree mask and the main tumor mask.

13. The cancer surgical margin status risk prediction system according to claim 12, characterized in that, The region determination module also includes: The planned resection plane determination module is used to map the bile duct tree into bile duct segments to be resected and bile duct segments to be preserved based on the three-dimensional bile duct-tumor model and the preset surgical plan; and to obtain the patient-specific planned resection plane by defining the spatial boundary between the bile duct segments to be resected and the bile duct segments to be preserved.

14. The cancer surgical margin status risk prediction system according to claim 12, characterized in that, The spatial characteristics include the volume of the main tumor, the maximum diameter of the main tumor, the number of planned bile duct resection sites, the minimum distance from the planned resection site to the boundary of the main tumor, the diameter of the bile duct at the resection site closest to the main tumor, the minimum diameter of the bile duct related to the resection site, the maximum diameter of the bile duct related to the resection site, and the ratio of the volume of the bile duct to be resected to the volume of the bile duct to be preserved.

15. The cancer surgical margin status risk prediction system according to any one of claims 12 to 14, characterized in that, The multi-scale image feature extraction module includes: The local feature extraction module is used to extract local multi-region image features from the medical image data through a preset three-dimensional image encoder; the preset three-dimensional image encoder is a Mamba-type three-dimensional vision network or a three-dimensional convolutional neural network. A global feature extraction module is used to extract global image features from the medical image data through a self-supervised learning 3D vision model; the sampling area of ​​the global image features is larger than the sampling area of ​​the local multi-region image features; the 3D vision model adopts the DINO model.

16. The cancer surgical margin status risk prediction system according to claim 15, characterized in that, The multimodal fusion prediction model includes a spatial mapping module, a fusion module, and a classification module; The prediction result determination module is used for: The spatial mapping module maps the spatial features, the local multi-region image features, and the global image features to a unified feature space. The fusion module fuses features in a unified feature space to obtain fused features; the fusion methods include residual fusion, gating fusion, attention fusion, or multilayer perceptron fusion. The classification module determines the probability of positive or negative margins, or the R0 / R1 auxiliary classification result for the planned resection plane specific to the patient, based on the fused features.

Images