Image-clinical characterization combined multi-endpoint prognosis evaluation method for jugular vein intrahepatic portal vena cava shunt
By employing cross-modal feature interaction and multi-task learning methods, combined with a small amount of manual annotation and deep learning features, the problems of high image feature extraction cost and poor model generalization in TIPS prognostic assessment are solved. This enables accurate assessment of multi-endpoint prognosis and improves the effectiveness of preoperative screening and treatment decisions for TIPS.
Patent Information
- Application Number
- CN202511867301.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-11
- Publication Date
- 2026-03-13
AI Technical Summary
In the prognostic assessment of transjugular intrahepatic portal shunt (TIPS) for portal hypertension caused by cirrhosis, existing technologies rely on costly manual annotation for image feature extraction, lack complementary information mining of image and clinical features in single-modal data modeling, resulting in poor generalization of prediction models, and traditional single-endpoint prediction ignores the association of multiple prognostic outcomes, thus limiting its clinical application value.
Using a small number of manually labeled datasets, combined with deep learning and radiomics features, a multi-endpoint prognostic assessment method was constructed through cross-modal feature interaction and multi-task learning to achieve the fusion of imaging and clinical information and assess multiple prognostic indicators such as survival after TIPS, portal vein pressure gradient, and hepatic encephalopathy.
While reducing annotation costs, it improves the generalization ability of the prediction model and the accuracy of multi-endpoint prognosis, providing comprehensive clinical support and offering a more effective basis for patient screening and treatment decisions before TIPS.
Smart Images
Figure CN121662351A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent medical technology, specifically to a multi-terminal prognostic assessment method for transjugular intrahepatic portal vein shunt based on combined imaging and clinical characterization. Background Technology
[0002] Portal hypertension caused by cirrhosis is a significant clinical problem worldwide, leading to serious complications such as refractory ascites and esophageal and gastric variceal bleeding. Transjugular intrahepatic portosystemic shunt (TIPS) can effectively reduce portal vein pressure, but it does not provide survival benefits for all patients and is often associated with postoperative complications such as hepatic encephalopathy. Therefore, accurate preoperative prognostic assessment is of great clinical importance.
[0003] Existing research has utilized machine learning and deep learning to model TIPS prognostic outcomes on imaging or clinical data. However, the extraction of imaging features relies heavily on meticulous manual annotation, leading to high costs for clinical applications. Methods based on single-modal data modeling lack in-depth mining of the correlation and complementary information between imaging and clinical features, often resulting in poor generalization of prediction models. Furthermore, traditional single-endpoint prediction ignores the potential associations between different prognostic outcomes, limiting model efficiency and clinical usability.
[0004] This invention proposes a multi-endpoint prognostic assessment method for transjugular intrahepatic portal vein shunt (TIPS) that combines imaging and clinical characterization. It achieves the acquisition of portal vein labels for all patients with only a small amount of manual portal vein annotation. By combining portal vein regional imaging information with structured clinical information, it completes cross-modal feature interaction and complementary information fusion. Under a unified framework, it simultaneously assesses multiple prognostic endpoints such as postoperative survival, portal vein pressure gradient, and hepatic encephalopathy, thereby providing more effective and comprehensive support for preoperative patient screening and treatment decisions for TIPS. Summary of the Invention
[0005] This invention provides a multi-terminal prognostic assessment method for transjugular intrahepatic portal vein shunt based on combined imaging and clinical characterization, characterized in that the method comprises:
[0006] Step 1: Collect a dataset of patients who underwent transjugular intrahepatic portal vein shunt (TIPS). For the original dataset Data review, screening, and a small amount of manual annotation were performed to create a dataset for TIPS prognosis. ;
[0007] Step 2: Construct a portal vein segmentation module with fewer annotations to obtain the dataset. Portal vein labels from preoperative CT scans of all patients.
[0008] Step 3: Extract deep learning features based on the patient's portal vein region. With radiomics characteristics ;
[0009] Step 4: Build a multimodal interactive representation learning module For deep learning features Radiomic characteristics Clinical characteristics Perform cross-modal interaction fusion to generate a unified representation ;
[0010] Step 5: Design a multi-endpoint prognostic prediction module for a unified representation of the decoded input. Includes, but is not limited to, postoperative survival prediction decoders Postoperative portal vein pressure gradient prediction decoder With postoperative hepatic encephalopathy prediction decoder ;
[0011] Step 6: Employ a multi-task learning strategy to learn the multimodal interaction representation module. and each prognostic decoder Conduct joint optimization training;
[0012] Step 7: Obtain multi-terminal prognostic prediction results, including post-TIPS survival, portal vein pressure gradient, and hepatic encephalopathy. .
[0013] Specifically, in step 1, acquiring patient data for TIPS treatment, reviewing and screening the data, and manually annotating a small amount of the portal vein region in the patient's preoperative CT scan, involves: acquiring preoperative abdominal CT images of the patient during the portal venous phase using a CT scanning device; collecting five types of preoperative structured clinical information, including baseline information, pressure-related indicators, blood and biochemical test indicators, clinical scores and grading, and previous surgical history; and collecting key postoperative results, including postoperative survival, portal vein pressure gradient values, and progression-free survival without hepatic encephalopathy, to form the original dataset. Cases with poor imaging quality or incomplete clinical data in the original dataset were excluded, and cases were randomly selected from the remaining cases. The portal vein region in the preoperative CT scan was manually annotated to create a dataset for TIPS prognosis. ;
[0014] Specifically, in step 2, the construction of the minimally labeled portal vein segmentation module obtains the dataset. Portal vein labels from preoperative CT scans of all patients were specifically determined by: whether the preoperative CT scans contained manual annotations, and then... Divided into labeled training set Annotated validation set and unlabeled set To expand the training samples, 3D CT data is converted into 2D images using axial slicing; the under-annotated portal vein segmentation module employs a sub-module based on semi-supervised semantic segmentation. Or a submodule based on a pre-trained visual model Implementation of any of the following schemes; for submodules The teacher-student moving average model is used as a semi-supervised training framework, and the DeepLabV3+ segmentation network is constructed as the student network. Teacher Network And a residual network is used as the encoder; during the training phase, using As a labeled training set As a verification set, and for Perform cross-validation partitioning, that is Divided into equal parts In each training round, one sample is selected to perform inference, while the rest are used as unlabeled samples for semi-supervised training. For each training round, the student network is optimized by a weighted combination of supervised loss and consistency regularization loss, with the total loss function being... The appropriate gradient optimizer is selected to update the network parameters; the teacher network weights are updated to the exponential moving average of the student network weights and stored in [the appropriate database / system]. The weights that are optimal for the above indicators are used for inference. For each round of inference, given a set of preoperative CT slices of the patient... , each slice Input Teacher Network Obtain the corresponding portal vein pixel-level prediction probability map. ,right Apply a fixed threshold Binarization is performed, and the images are reconstructed into a three-dimensional voxel-level mask in axial order. ;go through Rounds of training can yield unlabeled sets. The portal vein region labels for all cases are obtained, thus acquiring the portal vein annotation results from the preoperative CT scans of all patients; for submodules It is built using a general image segmentation basic model, which consists of an image encoder, a cue encoder, and a mask decoder; during the training phase, it uses... As a labeled training set As a verification set, the entire The image encoder and mask decoder are used as a test set for model inference; the cue encoder weights are fixed, the image encoder and mask decoder are fine-tuned using supervised loss, and the parameters are updated by selecting an appropriate gradient optimizer and saved. The model weights with the best performance index are used for inference; during the inference phase, the weights are adjusted based on the bounding box cue conditions. CT slices of each patient Inference is performed, and the results are reconstructed into a three-dimensional portal vein mask. ; and Based on the segmentation accuracy requirements and annotation complexity ( During the inference phase, bounding box hints should be provided to allow for trade-offs and choices.
[0015] Specifically, in step 3, the extraction of deep learning features based on the patient's portal vein region... With radiomics characteristics Specifically, given a dataset Preoperative CT images of patients and corresponding mask Using medical image pre-trained models Extracting portal vein voxel-level deep learning features Using radiomics feature extractors Extracting radiomics features ;
[0016] Specifically, in step 4, the construction of the multimodal interactive representation learning module... Specifically: It comprises three modules: multi-level feature interaction, complementary information enhancement, and feature discrimination optimization based on clinical priors. In the multi-level feature interaction module, based on radiomics features... The hierarchical structure (filter category - feature category - feature) is used for... Grouped by filter category - feature category, and then processed by feature encoder. Represented as fine-grained radiomics characteristics Deep learning features via feature encoder Obtain the encoded result And through cross-attention mechanism exist Fine-grained deep learning features are formed under the guidance of [the relevant authority / organization]. An adaptive aggregation method is used for... Aggregation yields coarse-grained radiomics characterization And using coarse-grained characterization to guide Aggregation to form coarse-grained deep learning features Fine-grained and coarse-grained features are concatenated to form full-grained deep learning features under multi-level interactions. and radiomics characteristics In the complementary information enhancement module, orthogonal constraint loss is used. right and The highest cosine similarity between them Orthogonalization constraints are applied to the features to suppress redundant information and enhance cross-modal complementarity; for the feature discrimination optimization module based on clinical priors, the clinical features are first... The five categories were grouped together using a feature encoder. Encode the data and obtain clinical representations through a self-interactive module. At the same time, and Each self-interactive module is input separately to obtain the representation after self-interaction. and Based on this, in clinical manifestations Guided by the cross-attention mechanism right and Perform cross-modal interaction to obtain the fused features after the interaction. and Subsequently, the feature set , , , and Adaptive aggregation methods are used to obtain the corresponding overall features. , , , and And splice them together to form a unified representation Because structured clinical features can explicitly encode biomedical prior information closely related to prognosis and generally exhibit stronger discriminative power than visual features in prognostic modeling, a unified representation constructed under the guidance of clinical features is therefore crucial. It has better prognostic ability;
[0017] Specifically, in step 5, the multi-endpoint prognostic prediction module is designed for decoding the unified representation of the input. Specifically, based on a unified representation Designing a postoperative survival prediction decoder based on feature dimensions, task attributes, and target application scenarios. A postoperative portal vein pressure gradient prediction decoder was designed based on the same principles. A postoperative hepatic encephalopathy prediction decoder was designed based on the same principles. ;
[0018] Specifically, in step 6, the multi-task learning strategy is used to learn the multimodal interaction representation module. and each prognostic decoder Joint optimization training is performed, specifically by integrating the multimodal interaction representation learning module. Encoding results As decoders for various prognostic tasks , , The input uses the real labels of the corresponding tasks as supervision information; a corresponding loss function is configured for each prognostic task, and is orthogonally constrained to the loss function. Together, they form a joint optimization objective; under the set loss weights, the multimodal interaction representation learning module and the prognostic task decoder are trained through multi-task joint optimization, and their total loss function is... Update model parameters based on the selected gradient optimizer, and save the results after training converges. and decoders for each prognostic task Model weights;
[0019] Specifically, in step 7, the results of multi-terminal prognostic prediction, such as post-TIPS survival, portal vein pressure gradient, and hepatic encephalopathy, are obtained. Specifically, this involves loading the multimodal interaction representation learning module obtained from step 6. and each prognostic decoder The weights are used to perform model inference, which is to say, deep learning features. Radiomic characteristics Clinical characteristics Enter to To generate a unified representation Subsequently Postoperative survival prediction results were obtained by decoding various prognostic decoders. Postoperative portal vein pressure gradient prediction results Postoperative hepatic encephalopathy prediction results . Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0021] Figure 1 Flowchart of a multi-terminal prognostic assessment method for transjugular intrahepatic portal vein shunt based on combined imaging and clinical characterization;
[0022] Figure 2 Flowchart for task integration of multi-endpoint prognostic prediction module. Detailed Implementation
[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0024] Current methods for preoperative prognostic assessment of transjugular intrahepatic portal vein shunt (TIPS) still face several bottlenecks: image feature extraction heavily relies on manual annotation, leading to high annotation costs; prediction frameworks based on single-modal data cannot utilize the complementarity between image and clinical information, limiting model reliability and generalization ability; and traditional single-endpoint prediction models struggle to achieve comprehensive prognostic assessment, resulting in insufficient clinical application value. This invention addresses these issues by innovatively proposing an intelligent preoperative prognostic assessment framework for TIPS based on minimally annotated segmentation, cross-modal feature interaction, and multi-task learning. This framework achieves a complete process from portal vein region feature extraction and image-clinical multimodal fusion to multi-endpoint prognostic prediction, enabling comprehensive patient prognostic assessment with minimal manual annotation and demonstrating significant clinical application potential. The specific steps are as follows:
[0025] S101: Dataset of patients who underwent transjugular intrahepatic portal shunt (TIPS) For the original dataset Data review, screening, and a small amount of manual annotation were performed to create a dataset for TIPS prognosis. .
[0026] Specifically, CT scans were used to acquire preoperative abdominal CT images of patients during the portal venous phase. Five types of preoperative structured clinical information were collected, including baseline information, pressure-related indicators, blood and biochemical test results, clinical scores and grading, and previous surgical history. Key postoperative outcomes were also collected, including postoperative survival, portal venous pressure gradient values, and progression-free survival without hepatic encephalopathy, forming the raw dataset. Cases with poor imaging quality or incomplete clinical data in the original dataset were excluded, and cases were randomly selected from the remaining cases. The portal vein region in the preoperative CT scan was manually annotated to create a dataset for TIPS prognosis. .
[0027] S102: Construct a portal vein segmentation module with few annotations to obtain the dataset. Portal vein label from preoperative CT scans of all patients.
[0028] Specifically, based on whether the preoperative CT scans have been manually annotated, the dataset is... Divided into labeled training set Annotated validation set and unlabeled set To expand the training samples, 3D CT data is converted into 2D images using axial slicing; the under-annotated portal vein segmentation module employs a sub-module based on semi-supervised semantic segmentation. Or a submodule based on a pre-trained visual model Implementation of any of the following schemes; for submodules The teacher-student moving average model is used as a semi-supervised training framework, and the DeepLabV3+ segmentation network is constructed as the student network. Teacher Network And a residual network is used as the encoder; during the training phase, using As a labeled training set As a verification set, and for Perform cross-validation partitioning, that is Divided into equal parts In each training round, one sample is selected to perform inference, while the rest serve as unlabeled samples for semi-supervised training. For each training round, the student network is optimized using a weighted combination of supervised loss and consistency regularization loss (e.g., Dice loss), with the total loss function being... The appropriate gradient optimizer (e.g., stochastic gradient descent) is selected to update the network parameters; the teacher network weights are updated to the exponential moving average of the student network weights and stored in [the appropriate database / system]. The weights that are optimal for the above indicators are used for inference. For each round of inference, given a set of preoperative CT slices of the patient... , each slice Input Teacher Network Obtain the corresponding portal vein pixel-level prediction probability map. ,right Apply a fixed threshold Binarization is performed, and the images are reconstructed into a three-dimensional voxel-level mask in axial order. ;go through Rounds of training can yield unlabeled sets. The portal vein region labels for all cases are obtained, thus acquiring the portal vein annotation results from the preoperative CT scans of all patients; for submodules It is built using a general image segmentation base model (e.g., SAM2), which consists of an image encoder, a cue encoder, and a mask decoder; during the training phase, it uses... As a labeled training set As a verification set, the entire The image encoder and mask decoder are used as a test set for model inference; the cue encoder weights are fixed, and the image encoder and mask decoder are fine-tuned using supervised losses (e.g., Dice loss and cross-entropy loss), with appropriate gradient optimizers (e.g., AdamW) selected to update the parameters, which are then saved. The model weights with the best performance index are used for inference; during the inference phase, the weights are adjusted based on the bounding box cue conditions. CT slices of each patient Inference is performed, and the results are reconstructed into a three-dimensional portal vein mask. ; and Based on the segmentation accuracy requirements and annotation complexity ( During the inference phase, bounding box hints need to be provided for trade-offs and choices.
[0029] S103: Extracting deep learning features based on the patient's portal vein region With radiomics characteristics .
[0030] Specifically, given a dataset Preoperative CT images of patients and corresponding mask Using medical image pre-trained models (e.g., MedicalNet) Extracts portal vein voxel-level deep learning features Using radiomics feature extractors (e.g., Pyradiomics) extracts radiomics features .
[0031] S104: Building a multimodal interactive representation learning module ,accomplish , Clinical characteristics Cross-modal interaction fusion to generate a unified representation .
[0032] Specifically, It comprises three modules: multi-level feature interaction, complementary information enhancement, and feature discrimination optimization based on clinical priors. In the multi-level feature interaction module, based on radiomics features... The hierarchical structure (filter category - feature category - feature) is used for... Grouped by filter category - feature category, and then processed by feature encoder. (For example, spiking neural networks) are represented as fine-grained radiomics features. Deep learning features via feature encoder (For example, a multilayer perceptron) obtains the encoded result And through cross-attention mechanism exist Fine-grained deep learning features are formed under the guidance of [the relevant authority / organization]. Adaptive aggregation methods (such as global attention pooling) are used to... Aggregation yields coarse-grained radiomics characterization And using coarse-grained characterization to guide Aggregation to form coarse-grained deep learning features Fine-grained and coarse-grained features are concatenated to form full-grained deep learning features under multi-level interactions. and radiomics characteristics In the complementary information enhancement module, orthogonal constraint loss is used. right and The highest cosine similarity between them Orthogonalization constraints are applied to the features to suppress redundant information and enhance cross-modal complementarity; for the feature discrimination optimization module based on clinical priors, the clinical features are first... The five categories were grouped together using a feature encoder. Encoding is performed using a neural network (e.g., a spiking neural network), and the clinical representation is obtained through a self-interactive module (e.g., the encoder of a Transformer). At the same time, and Each self-interactive module is input separately to obtain the representation after self-interaction. and Based on this, in clinical manifestations Guided by the cross-attention mechanism right and Perform cross-modal interaction to obtain the fused features after the interaction. and Subsequently, the feature set , , , and Adaptive aggregation methods are used to obtain the corresponding overall features. , , , and And splice them together to form a unified representation Because structured clinical features can explicitly encode biomedical prior information closely related to prognosis and generally exhibit stronger discriminative power than visual features in prognostic modeling, a unified representation constructed under the guidance of clinical features is therefore crucial. It has better prognostic ability.
[0033] S105: Design a multi-endpoint pre-prediction module for a unified representation of the decoded input. .
[0034] Specifically, based on a unified representation Designing a postoperative survival prediction decoder based on feature dimensions, task attributes, and target application scenarios. A postoperative portal vein pressure gradient prediction decoder was designed based on the same principles. A postoperative hepatic encephalopathy prediction decoder was designed based on the same principles. .
[0035] S106: Employing a multi-task learning strategy to learn the multimodal interaction representation module. and each prognostic decoder Conduct joint optimization training.
[0036] Specifically, the multimodal interaction representation learning module Encoding results As decoders for various prognostic tasks , , The input uses the real labels of the corresponding tasks as supervision information; a corresponding loss function is configured for each prognostic task, and is orthogonally constrained to the loss function. Together, they form a joint optimization objective; under the set loss weights, the multimodal interaction representation learning module and the prognostic task decoder are trained through multi-task joint optimization, and their total loss function is... Update model parameters based on the selected gradient optimizer (e.g., Adam), and save the results after training converges. and decoders for each prognostic task Model weights.
[0037] S107: Obtain multi-terminal prognostic prediction results such as post-TIPS survival, portal venous pressure gradient, and hepatic encephalopathy. .
[0038] Specifically, the multimodal interaction representation learning module trained in step S106 is loaded. and each prognostic decoder The weights are used to perform model inference, which is to say, deep learning features. Radiomic characteristics Clinical characteristics Enter to To generate a unified representation Subsequently Postoperative survival prediction results were obtained by decoding various prognostic decoders. Postoperative portal vein pressure gradient prediction results Postoperative hepatic encephalopathy prediction results .
[0039] For ease of understanding of the present invention, Figure 2The flowchart of the multi-endpoint prognostic prediction module task integration is shown, including but not limited to the construction of decoders for various prognostic tasks such as postoperative survival, portal vein pressure gradient, and hepatic encephalopathy prediction.
[0040] like Figure 2 The diagram shows the task integration process for the multi-endpoint prognostic prediction module, which specifically includes:
[0041] S201: Constructing a Postoperative Survival Prediction Decoder Specifically, a multilayer perceptron was used as the decoder for predicting postoperative survival. The basic network structure, settings Layers (e.g.) Linear fully connected layers are configured, and corresponding input dimensions are assigned to each layer. and output dimensions The output dimension of the last fully connected layer is set according to the actual application requirements (for example, in the survival regression prediction scenario, the output dimension can be 1). A non-linear activation function (such as ReLU or Sigmoid) is introduced after each fully connected layer to form a postoperative survival prediction decoder. .
[0042] S202: Constructing a Decoder for Predicting Postoperative Portal Vein Pressure Gradient Specifically, a multilayer perceptron is used as the basic component, and the main decoding path is constructed sequentially. Auxiliary decoding path and fusion prediction pathway ;in, Achieving a unified representation The decoding process, Incorporating pressure-related parameters as additional inputs can enhance their contribution to the prediction of portal vein pressure gradient. Fusion from and The features are then analyzed to generate the final prediction output; for each pathway Set separately Layers (e.g.) , , Linear fully connected layers are configured, and corresponding input dimensions are assigned to each layer. and output dimensions In particular, and The sum of the output dimensions of the last fully connected layer and The first layer has a consistent input dimension to support the input of concatenated features. The output dimension of the final fully connected layer is set according to the actual application requirements (for example, the output dimension can be 1 when used for pressure gradient regression prediction); a non-linear activation function (such as ReLU, Tanh) is introduced after each fully connected layer to form a postoperative portal vein pressure gradient prediction decoder. .
[0043] S203: Constructing a Decoder for Predicting Postoperative Hepatic Encephalopathy Specifically, a multilayer perceptron was used as a decoder for predicting postoperative hepatic encephalopathy. The basic network structure, settings Layers (e.g.) Linear fully connected layers are configured, and corresponding input dimensions are assigned to each layer. and output dimensions The output dimension of the last fully connected layer is set according to the actual application requirements (for example, in the regression prediction scenario of progression-free survival in hepatic encephalopathy, the output dimension can be 1). A non-linear activation function (such as ReLU or Sigmoid) is introduced after each fully connected layer to form a postoperative hepatic encephalopathy prediction decoder. .
[0044] This invention proposes a multi-terminal prognostic assessment method for transjugular intrahepatic portal vein shunt (TIPS) based on combined image-clinical representation. Through minimally annotated portal vein segmentation, cross-modal feature interaction, and multi-prognostic task co-learning, it achieves efficient fusion of preoperative CT and clinical information with minimal manual annotation, thereby completing a comprehensive multi-terminal prognostic assessment. This invention helps improve the effectiveness of preoperative patient screening and treatment decisions, and has significant clinical translational and application value.
Claims
1. A multi-terminal prognostic assessment method for transjugular intrahepatic portal vein shunt based on combined imaging and clinical characteristics, characterized in that... The method includes the following steps: Step 1: Collect patient data for transjugular intrahepatic portosystemic shunt (TIPS), including preoperative CT scans, structured clinical information, and postoperative outcomes, to form the raw dataset. ; for the dataset Data review and selection were conducted, and a small amount of manual annotation was performed on the portal vein region in the patients' preoperative CT scans to form a dataset oriented towards TIPS prognosis. ; Step 2: Construct a portal vein segmentation module with fewer annotations to obtain the dataset. The portal vein labeling module for preoperative CT scans of all patients is implemented using either of the following two schemes: a portal vein segmentation module based on semi-supervised semantic segmentation. Compared with the portal vein segmentation module based on a pre-trained large visual model ; Step 3: Extract deep learning features based on the patient's portal vein region. With radiomics characteristics ; Step 4: Build a multimodal interactive representation learning module For deep learning features Radiomic characteristics Clinical characteristics Cross-modal interaction fusion is performed, employing multi-level feature interaction, complementary information enhancement, and clinical prior-based feature discrimination optimization strategies to generate a unified representation. ; Step 5: Design a multi-endpoint prognostic prediction module for a unified representation of the decoded input. Includes a postoperative survival prediction decoder Postoperative portal vein pressure gradient prediction decoder With postoperative hepatic encephalopathy prediction decoder ; Step 6: Employ a multi-task learning strategy to learn the multimodal interaction representation module. and each prognostic decoder Conduct joint optimization training; Step 7: Obtain post-TIPS survival, portal vein pressure gradient, and multi-terminal prognostic prediction results for hepatic encephalopathy. .
2. The multi-terminal prognostic assessment method for transjugular intrahepatic portal vein shunt (TIPS) based on combined imaging and clinical characterization as described in claim 1, characterized in that... In step 1, the acquisition of patient data for TIPS treatment involves reviewing and filtering the data, and manually annotating a small portion of the portal vein region in the preoperative CT scan. Specifically, this includes acquiring preoperative abdominal CT images of the patient during the portal venous phase using a CT scanner, collecting five types of preoperative structured clinical information, including baseline information, pressure-related indicators, blood and biochemical test indicators, clinical scores and grading, and previous surgical history, and collecting key postoperative outcomes, including postoperative survival, portal vein pressure gradient values, and progression-free survival without hepatic encephalopathy, thus forming the original dataset. Cases with poor imaging quality or incomplete clinical data in the original dataset were excluded, and cases were randomly selected from the remaining cases. The portal vein region in the preoperative CT scan was manually annotated to create a dataset for TIPS prognosis. .
3. The multi-terminal prognostic assessment method for transjugular intrahepatic portosystemic shunt surgery based on combined imaging and clinical characterization as described in claim 1, characterized in that... In step 2, the construction of the minimally labeled portal vein segmentation module obtains the dataset. Portal vein labels from preoperative CT scans of all patients were specifically determined by: whether the preoperative CT scans contained manual annotations, and then... Divided into labeled training set Annotated validation set and unlabeled set To expand the training samples, 3D CT data is converted into 2D images using axial slicing; the under-annotated portal vein segmentation module employs a sub-module based on semi-supervised semantic segmentation. Or a submodule based on a pre-trained visual model Implementation of any of the following schemes; for submodules The teacher-student moving average model is used as a semi-supervised training framework, and the DeepLabV3+ segmentation network is constructed as the student network. Teacher Network And a residual network is used as the encoder; During the training phase, use As a labeled training set As a verification set, and for Perform cross-validation partitioning, that is Divided into equal parts In each training round, one sample is selected to perform inference, while the rest are used as unlabeled samples for semi-supervised training. For each training round, the student network is optimized by a weighted combination of supervised loss and consistency regularization loss, with the total loss function being... The appropriate gradient optimizer is selected to update the network parameters; the teacher network weights are updated to the exponential moving average of the student network weights and stored in [the appropriate database / system]. The weights that are optimal for the above indicators are used for inference. For each round of inference, given a set of preoperative CT slices of the patient... , each slice Input Teacher Network Obtain the corresponding portal vein pixel-level prediction probability map. ,right Apply a fixed threshold Binarization is performed, and the images are reconstructed into a three-dimensional voxel-level mask in axial order. ; go through Rounds of training can yield unlabeled sets. The portal vein region labels for all cases are obtained, thus acquiring the portal vein annotation results from the preoperative CT scans of all patients; for submodules It is built using a general image segmentation basic model, which consists of an image encoder, a cue encoder, and a mask decoder; during the training phase, it uses... As a labeled training set As a verification set, the entire The image encoder and mask decoder are used as a test set for model inference; the cue encoder weights are fixed, the image encoder and mask decoder are fine-tuned using supervised loss, and the parameters are updated by selecting an appropriate gradient optimizer and saved. The model weights with the best performance index are used for inference; During the inference phase, under the bounding box cues... CT slices of each patient Inference is performed, and the results are reconstructed into a three-dimensional portal vein mask. ; and Based on the segmentation accuracy requirements and annotation complexity ( During the inference phase, bounding box hints need to be provided for trade-offs and choices.
4. The multi-terminal prognostic assessment method for transjugular intrahepatic portal vein shunt (TIPS) based on combined imaging and clinical characterization as described in claim 1, characterized in that... In step 3, deep learning features are extracted based on the patient's portal vein region. With radiomics characteristics Specifically, given a dataset Preoperative CT images of patients and corresponding mask Using medical image pre-trained models Extracting portal vein voxel-level deep learning features Using radiomics feature extractors Extracting radiomics features .
5. The multi-terminal prognostic assessment method for transjugular intrahepatic portal vein shunt (TIPS) based on combined imaging and clinical characterization as described in claim 1, characterized in that... In step 4, the multimodal interactive representation learning module is constructed. Specifically: It comprises three modules: multi-level feature interaction, complementary information enhancement, and feature discrimination optimization based on clinical priors. In the multi-level feature interaction module, based on radiomics features... The hierarchical structure (filter category - feature category - feature) is used for... Grouped by filter category - feature category, and then processed by feature encoder. Represented as fine-grained radiomics characteristics Deep learning features via feature encoder Obtain the encoded result And through cross-attention mechanism exist Fine-grained deep learning features are formed under the guidance of [the relevant authority / organization]. ; Adaptive aggregation method is used for Aggregation yields coarse-grained radiomics characterization And using coarse-grained characterization to guide Aggregation to form coarse-grained deep learning features Fine-grained and coarse-grained features are concatenated to form full-grained deep learning features under multi-level interactions. and radiomics characteristics In the complementary information enhancement module, orthogonal constraint loss is used. right and The highest cosine similarity between them Orthogonalization constraints are applied to the features to suppress redundant information and enhance cross-modal complementarity; for the feature discrimination optimization module based on clinical priors, the clinical features are first... The five categories were grouped together using a feature encoder. Encode the data and obtain clinical representations through a self-interactive module. At the same time, and Each self-interactive module is input separately to obtain the representation after self-interaction. and Based on this, in clinical manifestations Guided by the cross-attention mechanism right and Perform cross-modal interaction to obtain the fused features after the interaction. and Subsequently, the feature set , , , and Adaptive aggregation methods are used to obtain the corresponding overall features. , , , and And splice them together to form a unified representation Because structured clinical features can explicitly encode biomedical prior information closely related to prognosis and generally exhibit stronger discriminative power than visual features in prognostic modeling, a unified representation constructed under the guidance of clinical features is therefore crucial. It has better prognostic ability.
6. The multi-terminal prognostic assessment method for transjugular intrahepatic portal vein shunt (TIPS) based on combined imaging and clinical characterization as described in claim 1, characterized in that... In step 5, the multi-endpoint prognostic prediction module is designed to decode the unified representation of the input. Specifically, based on a unified representation Designing a postoperative survival prediction decoder based on feature dimensions, task attributes, and target application scenarios. A postoperative portal vein pressure gradient prediction decoder was designed based on the same principles. A postoperative hepatic encephalopathy prediction decoder was designed based on the same principles. .
7. The multi-terminal prognostic assessment method for transjugular intrahepatic portal vein shunt (TIPS) based on combined imaging and clinical characterization as described in claim 1, characterized in that... In step 6, the multi-task learning strategy is used to learn the multimodal interaction representation module. and each prognostic decoder Joint optimization training is performed, specifically by integrating the multimodal interaction representation learning module. Encoding results As decoders for various prognostic tasks , , The input uses the real labels of the corresponding tasks as supervision information; a corresponding loss function is configured for each prognostic task, and is orthogonally constrained to the loss function. Together, they form a joint optimization objective; under the set loss weights, the multimodal interaction representation learning module and the prognostic task decoder are trained through multi-task joint optimization, and their total loss function is... Update model parameters based on the selected gradient optimizer, and save the results after training converges. and decoders for each prognostic task Model weights.
8. The multi-terminal prognostic assessment method for transjugular intrahepatic portosystemic shunt surgery based on combined imaging and clinical characterization as described in claim 1, characterized in that... In step 7, the multi-terminal prognostic prediction results of TIPS postoperative survival, portal vein pressure gradient, and hepatic encephalopathy are obtained. Specifically, this involves loading the multimodal interaction representation learning module obtained from step 6. and each prognostic decoder The weights are used to perform model inference, which is to say, deep learning features. Radiomic characteristics Clinical characteristics Enter to To generate a unified representation Subsequently Postoperative survival prediction results were obtained by decoding various prognostic decoders. Postoperative portal vein pressure gradient prediction results Postoperative hepatic encephalopathy prediction results .