A liver cancer postoperative recurrence risk score generation method and system based on step-by-step pathological-image residual error fusion

Abstract

The application discloses a liver cancer postoperative recurrence risk score generation method and system based on step-by-step pathological-image residual fusion, and belongs to the technical field of medical artificial intelligence, digital pathological analysis, medical image processing and tumor prognosis evaluation. The method obtains table feature representation and table branch risk score based on collected clinical data; obtains patient-level WSI pathological representation and WSI branch risk score based on WSI full-slice pathological images; fuses the table branch and the WSI branch to obtain Tabular+WSI anchor point risk; selects Top-K key layers according to a CT tumor mask, generates original cropped images, mask-enhanced cropped images and tumor mask occupation graphs around the lesion center to obtain CT branch risk score; performs scale alignment on the Tabular+WSI anchor point risk and the CT branch risk score, calculates a residual correction term, obtains a final postoperative recurrence risk score and outputs high-risk or low-risk stratification results. The method can improve the stability and accuracy of liver cancer postoperative recurrence risk prediction.

Description

Technical Field

[0001] This invention relates to the fields of medical artificial intelligence, tumor prognostic assessment, medical image processing, digital pathology analysis, and survival analysis. Specifically, it relates to a method and system for generating a postoperative recurrence risk score for hepatocellular carcinoma (HCC) patients to predict postoperative recurrence-free survival. This method comprehensively utilizes patient clinical pathology table data, whole-section pathological images, and continuous CT slices with tumor lesion masks. Through hierarchical WSI patient-level pathological characterization, 2.5D CT multi-channel cropping of the lesion center, clinical-pathological anchor risk fusion, and CT image residual complementary correction, it outputs a patient-level postoperative recurrence risk score and risk stratification results, which can be used for postoperative recurrence risk assessment, follow-up strategy development, and as an auxiliary reference for clinical decision-making. Background Technology

[0002] Hepatocellular carcinoma is the main pathological type of primary liver cancer. Even after radical resection and other treatments, postoperative recurrence remains a key issue affecting long-term prognosis. Clinically, it is usually necessary to identify patients at high risk of recurrence early after surgery in order to develop closer follow-up strategies and adjuvant therapy plans. Therefore, patient-level risk prediction for recurrence-free survival has clear clinical application value.

[0003] Currently, the prediction of postoperative recurrence risk in liver cancer mainly relies on clinical pathological variables, survival analysis models, pathological WSI image analysis models, CT radiomics models, and multimodal fusion models. However, existing methods still have shortcomings in the utilization of multi-source information, patient-level pathological characterization, extraction of local imaging features of lesions, and multimodal complementary fusion.

[0004] First, existing clinical pathology table prediction methods mainly rely on structured clinical variables for modeling, such as patient age, tumor size, alpha-fetoprotein (AFP), microvascular invasion (MVI), and tumor staging indicators. These methods typically employ traditional statistical learning or machine learning techniques such as Cox proportional hazards models, LASSO-Cox, Ridge regression, random forests, ExtraTrees, or gradient boosting trees to build recurrence risk models. While these methods offer relatively stable modeling processes and strong interpretability, they fail to fully utilize the tumor heterogeneity, histological morphology, and spatial texture information contained in pathological sections and CT images due to their reliance on clinical pathology table variables. Therefore, their ability to characterize the recurrence risk after surgery for complex liver cancer is limited.

[0005] Second, existing WSI pathology image analysis methods typically segment whole-slice pathology images into local patches before feature extraction and patient-level aggregation, such as patch-CNN, Attention-MIL, CLAM, and TransMIL. These methods can extract cell morphology, tissue structure, and tumor microenvironment-related features from pathology images. However, WSI images are large in size and contain significant local noise. Without a progressive processing and stable aggregation mechanism from patch, region, slice to patient level, patient-level analysis can easily lead to unstable patient-level risk representation. Furthermore, relying solely on WSI pathology images makes it difficult to fully integrate diagnostic and treatment information from clinical pathology table variables and lesion spatial information from CT images.

[0006] Third, existing CT image prediction methods mostly employ artificial radiomics features or directly use 2D or 3D convolutional networks for modeling, such as radiomics-Cox, radiomics-SVM, 2D CNN, 3D ResNet, and 3D DenseNet. These methods can utilize tumor density, shape, texture, and spatial distribution information in CT images; however, traditional radiomics methods rely on manually designed features, resulting in limited generalization ability. Training deep models directly using complete CT images or complete 3D volume data requires a large sample size and computational resources, and easily introduces a large amount of background information unrelated to tumor recurrence. For small-sample hepatocellular carcinoma postoperative recurrence prediction tasks, existing methods often struggle to achieve a balance between preserving local lesion context information and controlling model complexity.

[0007] Fourth, existing multimodal fusion methods often employ direct splicing of original features, simple averaging of risk scores, or ordinary integration methods, which easily overlook the primary and secondary relationships and differences in risk scales between different modalities. Especially when clinical pathology tables, WSI pathology images, and local CT lesion images are used together in modeling, if a stable clinical-pathological backbone risk cannot be constructed first, and then constrained residual correction is performed using CT images, problems such as dilution of effective modal contributions, unstable fusion results, and overfitting under small sample conditions may occur. Summary of the Invention

[0008] To address the shortcomings of existing methods for predicting postoperative recurrence risk in liver cancer, such as unstable patient-level pathological representation, insufficient utilization of local CT lesion information, unclear multimodal risk fusion mechanisms, and inconsistent output scales across different risk branches, this invention proposes a method and system for generating postoperative recurrence risk scores for liver cancer based on hierarchical pathology-image residual fusion. This method establishes a hierarchical representation and risk level complementarity mechanism around three types of data: clinical forms, WSI pathological images, and local CT lesion images, focusing on solving the following technical problems:

[0009] 1. How to aggregate WSI whole-section pathological images from local patches to patient-level pathological features step by step, reducing the interference of invalid patches, local noise and single-slice bias on patient-level recurrence risk modeling;

[0010] 2. How to use CT tumor masking to select key layers and generate a 2.5D multi-channel local cropping representation around the lesion center, while preserving lesion texture, boundaries and surrounding tissue context information and reducing background redundancy;

[0011] 3. How to first construct clinical-pathological anchor risk from clinical pathology tables and WSI pathological features, and then use CT branches as residual supplements to correct the anchor risk after scale alignment, thereby forming a three-modal fusion mechanism with clear primary and secondary elements and consistent risk scale.

[0012] To address the aforementioned technical problems, this invention provides a method for generating a postoperative recurrence risk score for liver cancer based on stepwise pathology-imaging residual fusion, comprising the following steps:

[0013] S1. Acquire clinical pathology table data, WSI whole-slice pathology images, CT serial slices, CT tumor masks, and recurrence-free survival time and recurrence status used in the training or validation phase for the same patient, and align the patient-level multimodal data according to the patient ID to form aligned patient-level multimodal samples.

[0014] S2 performs missing value imputation and standardization on clinical pathology table data, and generates clinical pathology table feature representation and table branch risk score through risk-related modeling.

[0015] S3 performs tissue region detection, patch segmentation, and patch feature extraction on WSI whole-slice pathological images, and aggregates them step by step at the region level, slice level, and patient level to generate patient-level WSI pathological features and WSI branch risk scores.

[0016] S4: Input clinical pathology table features, table branch risk scores, patient-level WSI pathological features and WSI branch risk scores into the clinical-pathology anchor fusion module to generate Tabular+WSI anchor risk.

[0017] S5. Based on the CT tumor mask, the tumor mask area of ​​each CT slice is statistically analyzed, and the Top-K key slices with the largest area are selected. For each key slice, the lesion center is determined according to the bounding box of the tumor mask. Square cropping and size standardization are performed around the lesion center to generate the original lesion center cropped image, the mask-enhanced cropped image, and the tumor mask occupancy image. Figure 3 CTcrop-like characterization, and further generate patient-level CT feature characterization and CT branch risk score;

[0018] S6, scale-align the Tabular+WSI anchor point risk and CT branch risk scores, calculate the residual correction term of CT branch relative to anchor point risk, and correct the anchor point risk with a fixed residual correction coefficient to obtain the final patient-level postoperative recurrence risk score.

[0019] S7 outputs risk stratification results of high or low risk based on the final patient-level postoperative recurrence risk score.

[0020] This invention also provides a system for generating a risk score for postoperative recurrence of liver cancer, comprising a data acquisition module, a patient-level multimodal data alignment module, a clinical table feature processing module, a WSI patient-level characterization module, a CT lesion center cropping module, a clinical-pathological anchor point fusion module, a residual complementary fusion module, and a risk score output module. Each module corresponds to one of the steps described above and is sequentially connected via a data stream.

[0021] Compared with the prior art, the present invention has at least the following beneficial effects:

[0022] 1. This invention uses a stepwise WSI patient-level representation module to aggregate WSI images from patch level, region level, and slice level to patient-level representation, avoiding simple averaging or one-time attention summarization of all patches. This can reduce the impact of invalid patches, local noise, tissue folding, and single-slice bias on patient-level recurrence risk prediction.

[0023] 2. This invention uses a 2.5D CT multi-channel cropping module at the lesion center to select key layers based on the tumor mask and generate three types of representations—Raw crop, Masked crop, and Mask map—around the center of the tumor bounding box. This preserves the local texture, boundary, and surrounding tissue context of the lesion while reducing background redundancy in complete CT volume data and the risk of overfitting under small sample conditions.

[0024] 3. This invention uses a clinical-pathological anchor fusion module to first construct a stable backbone risk using clinical pathology table branches and WSI pathology branches, so that the interpretable information of clinical variables and pathological morphological information complement each other, avoiding the insufficient predictive ability of single-modality risk.

[0025] 4. This invention uses a residual complementary fusion module to correct the risk level of Tabular+WSI anchor point risk by using CT branches as image residual supplements, after scale alignment. This clarifies the primary and secondary relationship between the clinical-pathological main branch and the CT supplementary branch, avoiding the dilution of effective information caused by direct splicing or simple averaging of original features.

[0026] 5. In the training and validation process, this invention uses patient-level data partitioning to ensure that data from the same patient will not appear in both the training and validation sets at the same time; missing value imputation, standardization, dimensionality reduction, risk mapping, and model training are all completed within the training data range, which helps to reduce the risk of information leakage and improve the reliability of generalization evaluation on patients who have not been seen. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the overall process of generating a risk score for postoperative recurrence of liver cancer according to the present invention.

[0028] Figure 2 This is a schematic diagram of the structure of the liver cancer postoperative recurrence risk score generation system of the present invention;

[0029] Figure 3 This is a flowchart of the clinical pathology table feature processing of the present invention;

[0030] Figure 4 This is a flowchart of the step-by-step WSI patient-level pathological characterization process of the present invention;

[0031] Figure 5 This is a flowchart of the clinical-pathological anchor point fusion process of the present invention;

[0032] Figure 6 This is a flowchart of the 2.5D CT cropping process for the lesion center in this invention;

[0033] Figure 7 This is a flowchart of the residual complementary fusion and risk scoring output of the present invention. Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. All equivalent substitutions, parameter adjustments, model replacements, or process equivalent modifications made based on the technical concept of the present invention should fall within the scope of protection of the present invention.

[0035] This embodiment provides a hierarchical pathology-image residual fusion network, SPRF-Net, for predicting the risk of postoperative recurrence in liver cancer. The system uses a single patient as the basic processing unit. After obtaining the patient's clinical pathology table data, WSI whole-slice pathology images, CT serial slices, and tumor masks, it sequentially completes multimodal data alignment, clinical table risk modeling, WSI patient-level pathological characterization, clinical-pathology anchor point risk construction, 2.5D cropping of CT lesion centers, residual complementary fusion, and final risk score output. The overall process is as follows: Figure 1 As shown, the system structure is as follows Figure 2 As shown.

[0036] The meanings of the core variables and symbols involved in this embodiment are as follows:

[0037] Indicates the first Patient-level multimodal samples from one patient; Representing clinical pathology table data; This represents the characteristics of a clinical pathology table. This indicates the risk score for each branch of the table; This represents a whole-section pathological image from WSI. This indicates patient-level WSI pathological features; This indicates the risk score for the WSI branch.

[0038] Represents a CT image sequence; Indicates the CT tumor mask sequence; This represents patient-level CT feature characterization; Indicates the CT branch risk score; This represents the Tabular+WSI anchor point risk score; This indicates the risk of the normalized Tabular+WSI anchor point; This represents the normalized risk of CT branching. This represents the CT residual correction term; Represents the residual correction factor; This represents the final patient-level postoperative recurrence risk score.

[0039] Acquire and align patient-level multimodal data

[0040] First, the system acquires multimodal data from the same patient, including clinicopathological table data, WSI whole-section pathological images, CT serial sections, CT tumor masks, and recurrence-free survival time and recurrence status used during the training or validation phase. For the first... For each patient, their patient-level multimodal samples are represented as follows:

[0041]

[0042] in, Indicates the first Clinical pathology data for each patient This represents a whole-section pathological image from WSI. Represents a CT image sequence. Indicates the CT tumor mask sequence. Indicates relapse-free survival time. This indicates the recurrence status. When actually predicting new patients, the system only needs to input clinicopathological table data, WSI images, CT images, and tumor masks. Recurrence-free survival time and recurrence status are only used in the training and performance evaluation phases.

[0043] The system performs consistency matching on different modalities based on patient ID, ensuring that clinical tables, WSI images, CT images, tumor masks, and survival labels for the same patient correspond to each other, thus forming patient-level multimodal samples. This step outputs aligned patient-level multimodal data, which serves as the unified input for subsequent branch processing. During the training and validation phases, patient-level fixed-fold partitioning is used, ensuring that no single modality of data from the same patient appears simultaneously in the training and validation sets.

[0044] Process clinical pathology tabular data to generate clinical risk characterization

[0045] like Figure 3 As shown, the system reads the first... Clinical pathology data for each patient. Clinical pathology data includes, but is not limited to, patient age, tumor size, alpha-fetoprotein (AFP), microvascular invasion (MVI), tumor stage, inflammation-related markers, and other structured variables associated with postoperative recurrence of liver cancer. The original clinicopathological variables for each patient are represented as follows:

[0046]

[0047] in, Indicates the number of clinicopathological variables. These represent different clinicopathological variables.

[0048] For clinical table data, the system first imputes missing values ​​to reduce the impact of missing clinical variables on the model input; then, it standardizes variables of different dimensions to ensure that different clinical indicators are within a comparable numerical range; finally, it obtains the clinical pathology table feature representation through risk-related modeling.

[0049]

[0050] in, This represents the preprocessing and characterization functions for clinical pathology tables. This represents the clinical pathology table features. Further, a survival risk prediction model is used to obtain the risk scores for each branch of the table.

[0051]

[0052] in, This represents the risk prediction function for clinical pathology tables. This represents the patient's baseline clinical risk score. This step outputs a clinicopathological risk profile. and table branch risk score This serves as the foundational input for subsequent clinical-pathological anchor point fusion.

[0053] In one specific embodiment, missing value imputation, standardization, and risk modeling are all performed by fitting the corresponding parameters only within the training fold, and then applying the fixed parameters to validation patients or patients to be predicted, in order to avoid leakage of validation information. Risk modeling function This can be achieved using Cox proportional hazards models, ridge regression survival models, tree models, or other models that can output patient-level risk scores.

[0054] Process WSI whole-section pathological images to generate patient-level WSI pathological features

[0055] like Figure 4 As shown, the system reads the first... WSI whole section pathology images of one patient Because WSI images are large, high-resolution, and contain numerous local image patches, directly inputting the entire image for risk prediction is easily affected by blank backgrounds, staining differences, tissue folds, low-quality areas, and local noise patches. Therefore, the system employs a hierarchical WSI patient-level characterization module to process WSI images step-by-step from local to global.

[0056] First, the system performs tissue region detection on the WSI image, removing blank background areas and obviously invalid areas, retaining only areas containing valid pathological tissue. Then, within the valid tissue area, local image patches are divided according to a fixed size and a fixed step size, resulting in a patch set.

[0057]

[0058] in, Indicates the first A pathological image block, This indicates the number of valid patches for that patient. Patches with excessively high blank ratios, small tissue areas, or poor image quality are removed by the system to reduce the interference of low-quality regions on subsequent risk prediction.

[0059] Subsequently, the system uses a WSI patch feature extraction network or a pre-trained pathological feature encoder to extract features from each patch, obtaining patch-level features:

[0060]

[0061] in, This represents the WSI patch feature extraction network. Indicates the first The feature vector of each patch. This feature vector is used to express information such as cell morphology, tissue structure, tumor region, stroma region, necrotic region, and tumor microenvironment in a local pathological image patch.

[0062] After obtaining patch-level features, the system does not simply average all patch features, but instead aggregates them step by step at the region, slice, and patient levels. First, patch features that are spatially close, have similar tissue regions, or similar semantic features are aggregated at the region level to obtain region-level pathological representations.

[0063]

[0064] Then, multiple region-level characteristics in the same WSI slide are aggregated at the slide level to obtain slide-level pathological characteristics:

[0065]

[0066] For cases where one or more WSI slides exist for the same patient, the system further aggregates the slide-level characterizations at the patient level to obtain patient-level WSI pathological characterizations:

[0067]

[0068] in, , and These represent aggregation operations at the region, slice, and patient levels, respectively. This represents the final patient-level WSI pathological characterization. In one embodiment, the above aggregation operation can be implemented using attention-weighted aggregation, or using average pooling, gated pooling, or top-k screening aggregation based on regional risk contribution scores.

[0069] Finally, the system calculates the WSI branch risk score based on patient-level WSI pathological characteristics:

[0070]

[0071] in, This represents the WSI pathology risk prediction function. This represents the pathological risk score output by the WSI branch. This step outputs patient-level WSI pathological features. and WSI risk score .

[0072] By integrating clinical form risk and WSI pathological risk, a clinical-pathological anchor risk is generated.

[0073] like Figure 5 As shown, after obtaining the clinicopathology form risk score... and WSI branch risk score Subsequently, the system constructs Tabular+WSI anchor risk through the clinical-pathological anchor fusion module. The inputs to this step are clinical-pathological tabular feature representation, tabular branch risk score, patient-level WSI pathological characterization, and WSI branch risk score, and the output is the clinical-pathological anchor risk score.

[0074] Unlike simply concatenating tabular and WSI features, the clinical-pathological anchor fusion module uses the clinical-pathological tabular branch as a relatively stable basic risk branch and the WSI pathology branch as supplementary pathological information to enhance and correct the basic clinical risk. The fusion process is represented as follows:

[0075]

[0076] in, This represents the Tabular+WSI anchor risk score. This indicates the risk score in the clinical pathology form. Indicates WSI pathology risk score, This indicates the supplementary risk correction item for the WSI branch relative to the table branch. Indicates the gating weight, and This represents the fusion coefficient.

[0077] The output of this step This represents a stable clinical-pathological backbone risk, used for subsequent residual complementary fusion with CT branches. By first constructing clinical-pathological anchor risks, information dilution caused by indiscriminately splicing clinical tables, WSI, and CT features in subsequent three-modal fusion can be avoided.

[0078] Process CT images and tumor masks to generate 2.5D CT lesion clipping features.

[0079] like Figure 6 As shown, the system reads the first... A series of CT scans of a patient and their corresponding tumor masks. The CT image sequence is represented as follows:

[0080]

[0081] The corresponding tumor mask sequence is represented as follows:

[0082]

[0083] in, Indicates the first CT scan level, This indicates the tumor mask at this level. Indicates the number of CT slices.

[0084] First, the system calculates the tumor mask area for each CT slice:

[0085]

[0086] in, Indicates the first The tumor mask area on each CT scan was determined. Empty mask layers were removed, and the tumors were sorted by mask area from largest to smallest, selecting the largest mask layer. Each layer is considered a key lesion layer. In this embodiment, .

[0087] For each selected key layer, the system determines the lesion center based on the tumor mask bounding box:

[0088]

[0089] in, , , , These represent the coordinate ranges of the bounding box of the tumor mask. Indicates the coordinates of the lesion center.

[0090] Subsequently, the system settings include trimming the side length of the lesion center:

[0091]

[0092] in, This indicates the side length to be cut off at the center of the lesion. This indicates the proportion of the surrounding context expansion. In this embodiment, The system is based on Cut a square around the center and adjust the cutting area uniformly. The size is adjusted to obtain a compact local imaging area of ​​the lesion.

[0093] For each patient, the system generates three types of CT crop representations:

[0094]

[0095]

[0096]

[0097] in, This represents a cropped image of the original lesion center. This represents a cropped image of the lesion enhanced by combining a tumor mask. This represents the tumor mask occupancy map. Subsequently, the system aggregates crop features from multiple key layers to form patient-level CT crop features:

[0098]

[0099] in, This represents the CT crop feature aggregation function. Furthermore, the system standardizes and performs PCA dimensionality reduction on the CT features within the training layer to obtain a compact CT representation, and then uses the CT risk prediction model to obtain the CT branch risk score.

[0100]

[0101] in, This represents the CT branch risk prediction function. This represents the risk score output by the CT branch. This step outputs patient-level CT feature representation. and CT branch risk score .

[0102] Residual complement fusion is performed to obtain the final relapse risk score.

[0103] like Figure 7 As shown, in obtaining Tabular+WSI anchor point risk and CT branch risk Then, the system calculates the final patient-level relapse risk score through the residual complementary fusion module.

[0104] The residual complementarity fusion module does not re-perform the fusion of clinical-pathological tabular features and WSI pathological features. Instead, it uses the Tabular+WSI anchor risk output by the preceding clinical-pathological anchor fusion module as input. The clinical-pathological anchor fusion module is responsible for constructing a stable clinical-pathological backbone risk, while the residual complementarity fusion module further utilizes the local imaging risk information provided by the CT 2.5D lesion clipping branches to supplement and correct the backbone anchor risk. The two constitute a continuous two-level fusion mechanism.

[0105] Since the Tabular+WSI branch and the CT branch come from different model families, their output risk scores are not entirely consistent on a numerical scale. Therefore, this embodiment fits a percentile mapping function of the risk scores in the training fold or historical training queue, and applies this mapping function to validation patients or new patients after fixing it. The mapping process is expressed as follows:

[0106]

[0107]

[0108] in, This represents the percentile-rank normalization function representing the risk score fitted within the training fold. This represents the normalized Tabular+WSI anchor point risk. This represents the normalized CT branch risk. This transformation uses only the model-predicted risk scores to align different model output scales, without using patient outcome labels.

[0109] After scale alignment, the system calculates the residual correction term of the CT branch relative to the anchor risk, using Tabular+WSI anchor risk as a benchmark:

[0110]

[0111] in, This represents the residual supplementary information provided by the CT 2.5D lesion trimming branch relative to the Tabular+WSI anchor point risk. When the risk predicted by the CT branch is higher than the anchor point risk... A positive value indicates that the CT branch has made an upward adjustment to the patient's risk; when the risk predicted by the CT branch is lower than the anchor point risk... A negative value indicates that the CT branch has downgraded the risk assessment for the patient.

[0112] Subsequently, the system uses a fixed residual correction coefficient. After adjusting for anchor point risk, the final patient-level recurrence risk score is obtained:

[0113]

[0114] in, This represents the final patient-level postoperative recurrence risk score. This indicates the strength of the correction for anchor point risk by the CT residual supplementary information. To avoid overfitting risks and validation bias caused by dynamic weight fitting under small sample conditions, this embodiment does not perform dynamic weight search or select candidate models based on the validation set, but instead uses fixed residual correction coefficients.

[0115] In this embodiment, we take:

[0116]

[0117] therefore:

[0118]

[0119] The above formula shows that the CT branch does not directly replace the clinical-pathological anchor risk, but rather serves as a residual supplement to correct the anchor risk after scale alignment, thereby improving the stability of the trimodal risk score.

[0120] Output patient-level recurrence risk scores and risk stratification results

[0121] The system ultimately outputs a patient-level postoperative recurrence risk score. A higher score indicates a higher risk of postoperative recurrence; a lower score indicates a relatively better prognosis. Patients can be further divided into high-risk and low-risk groups based on preset thresholds, the median risk value of the training cohort, or clinically set thresholds.

[0122] During the training and validation phases, the system can calculate the OOF C-index based on the OOF risk scores of all validation patients to evaluate the model's ability to rank patients' recurrence risk; simultaneously, it can also... Kaplan-Meier relapse-free survival curves were plotted, and the model's risk stratification capability was evaluated using the log-rank test. In practical applications, the system can provide patients' risk scores and risk stratification results to clinicians to assist in determining postoperative follow-up frequency, aiding in treatment decisions, and providing reference for individualized prognostic management.

[0123] Model Training and Validation Examples

[0124] In this embodiment, the experimental data came from a cohort predicting recurrence-free survival after hepatocellular carcinoma surgery. Each patient was analyzed at the patient level, incorporating structured clinicopathological tables, WSI whole-slice pathological images, CT serial slices, and corresponding tumor lesion masking data. Postoperative recurrence-free survival time and recurrence status were recorded as survival analysis labels. After consistency matching of clinical tables, WSI features, CT features, and survival labels using patient IDs, this embodiment yielded a total of 108 patient-level samples for trimodal modeling and validation. Among these, 24 patients experienced recurrence events, and 84 patients were either recurrent or censored. The median observable follow-up or outcome time was 12.68 months.

[0125] To avoid the simultaneous appearance of multimodal data from the same patient in both the training and validation sets, this embodiment employs patient-level fixed 4-fold cross-validation for model training and validation. Each fold contains 27 patients and 6 recurrence events. Experiments were repeated with three random seeds of 42, 52, and 62 patients. The average OOF C-index was calculated using the out-of-fold prediction results of all validation fold patients to evaluate the model's risk ranking ability on patients without prior observation. Throughout the project, clinical tabular data, CT features, WSI features, and survival labels were aligned by patient ID, and patient-level fixed fold partitioning was consistently used for validation.

[0126] To further verify the contribution of each modality and module to the final prediction performance, this experiment included ablation experiments with single-table, single-WSI, single-CT, table+WSI, and three-modal fusion models. The experimental results are shown in the table below.

[0127] Clinical Cox 0.694147 WSI 0.661171 CT 0.745260 Tabular+WSI 0.810800 Tabular + WSI + CT 0.875103

[0128] As shown in the table above, the trimodal SPRF-Net proposed in this invention achieves the highest OOF C-index under all three random seeds, with an average OOF C-index of 0.875103, which is superior to the single-table, single-WSI, single-CT, and table + WSI bimodal models. This indicates a significant complementarity between clinical pathology information, WSI pathological image information, and CT lesion local image information. The average OOF C-index of the table + WSI model is improved to 0.810800, indicating that the clinical pathology table features and WSI pathological image features can effectively complement each other, and the clinical-pathology anchor fusion module can construct a more stable main risk branch. After further introducing 2.5D CT tumor center clipping features and fusing them through the residual complementary fusion module, the average OOF C-index increased to 0.875103, which is 0.064303 higher than the tabular + WSI bimodal model. This indicates that the CT 2.5D lesion clipping branch can provide additional imaging information beyond clinicopathological and WSI information, and effectively corrects the risk results of Tabular + WSI anchor points.

[0129] The high-risk and low-risk groups, derived from the patient-level recurrence risk score output by SPRF-Net, showed significant differences in recurrence-free survival. The recurrence-free survival curves of the high-risk group were generally lower than those of the low-risk group, indicating that patients in the high-risk group were more prone to recurrence during postoperative follow-up, and their recurrence-free survival rate declined more rapidly. Conversely, the recurrence-free survival curves of the low-risk group remained at a relatively high level, indicating that this group had better postoperative prognosis. Log-rank test results showed that the difference in recurrence-free survival curves between the high-risk and low-risk groups was statistically significant, further demonstrating that the postoperative recurrence risk score output by this invention has good patient risk stratification capabilities.

[0130] This invention also provides a system for generating a risk score for postoperative recurrence of liver cancer, comprising the following modules:

[0131] The data acquisition module is used to acquire clinical pathology table data, WSI whole slide pathology images, CT serial slides, CT tumor masks, and recurrence-free survival time and recurrence status of the same patient during the training or validation phase.

[0132] The patient-level multimodal data alignment module is used to perform consistency matching of clinical pathology tabular data, WSI whole-slice pathology images, CT serial slices, CT tumor masks, and survival labels according to patient ID, forming patient-level multimodal samples.

[0133] The clinical table feature processing module is used to perform missing value imputation, standardization, and risk-related modeling on clinical pathology table data, and output clinical pathology table feature representations and table branch risk scores.

[0134] The WSI patient-level characterization module is used to perform tissue region detection, patch segmentation, patch feature extraction, region-level aggregation, slice-level aggregation and patient-level aggregation on WSI whole-slice pathological images, and output patient-level WSI pathological characterization and WSI branch risk score.

[0135] The CT lesion center cropping module is used to select the Top-K key layers based on the CT tumor mask, determine the lesion center, perform square cropping and size uniformity around the lesion center, and generate three types of CTcrop representations: Raw crop, Masked crop, and Mask map. It outputs patient-level CT feature representations and CT branch risk scores.

[0136] The clinical-pathology anchor fusion module is used to fuse the clinical-pathology table risk score with the WSI branch risk score and output the Tabular+WSI anchor risk.

[0137] The residual complementary fusion module is used to scale-align Tabular+WSI anchor point risk and CT branch risk scores, calculate CT residual correction terms, and output the final patient-level postoperative recurrence risk score.

[0138] The risk score output module is used to output stratified results of high or low risk based on the final patient-level postoperative recurrence risk score.

[0139] The present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory, wherein the computer program, when executed by the processor, implements the steps of the above-described method for generating a risk score for postoperative recurrence of liver cancer.

[0140] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method for generating a risk score for postoperative recurrence of liver cancer.

[0141] Example

[0142] The overall implementation process of the SPRF-Net algorithm for predicting the risk of postoperative recurrence of liver cancer is as follows:

[0143] The SPRF-Net model is defined, including a clinical pathology table feature processing module, a hierarchical WSI patient-level representation module, a clinical-pathology anchor point fusion module, a lesion center 2.5D CT cropping module, and a residual complementary fusion module.

[0144] Load the patient's clinical pathology table data, fill in missing values, standardize clinical variables of different dimensions, extract clinical pathology table features, and obtain table feature representations;

[0145] Load the patient's whole-section WSI pathological image, perform tissue region detection on the WSI image, remove invalid background regions, divide the valid tissue region into multiple local patches, extract features from each patch, and perform region-level aggregation, slice-level aggregation and patient-level aggregation on the patch features to obtain the patient-level WSI representation.

[0146] The clinical pathology table features and patient-level WSI representations are input into the clinical-pathology anchor fusion module. The WSI pathology representations are used to supplement and correct the basic clinical pathology risks, and generate a Tabular+WSI anchor risk score.

[0147] Load the patient's CT image sequence and its corresponding tumor mask, count the tumor mask area of ​​each CT layer, select the Top-K key layers with the largest tumor mask area, determine the lesion center position based on the tumor mask bounding box, perform square cropping with surrounding context around the lesion center, uniformly adjust the cropped images to a fixed size, generate Raw crop, Masked crop and Mask map respectively, aggregate the crop features of multiple key layers to obtain CT feature representation, and calculate CT branch risk score;

[0148] Input the Tabular+WSI anchor risk score and the CT branch risk score, perform scale alignment or in-fold rank normalization on the two risk scores, use the CT branch as residual complementary information to supplement and correct the Tabular+WSI anchor risk, and calculate the final patient-level recurrence risk score.

[0149] For new patients to be predicted, their clinical pathology table data, WSI images, CT images and tumor masks are input. The data are then processed sequentially through table feature processing, progressive WSI patient-level characterization, clinical-pathology anchor point fusion, 2.5D CT cropping of lesion centers and residual complementary fusion, and output patient-level postoperative recurrence risk scores and risk stratification results.

Claims

1. A method for generating a postoperative recurrence risk score for liver cancer based on hierarchical pathology-imaging residual fusion, characterized in that, Includes the following steps: S1. Acquire clinical pathology table data, WSI whole-slice pathology images, CT continuous slices and CT tumor mask of the same patient, and perform patient-level consistency matching on the clinical pathology table data, the WSI whole-slice pathology images, the CT continuous slices and the CT tumor mask according to the patient ID to form patient-level multimodal samples. S2, perform missing value imputation and standardization on the clinical pathology table data, and generate clinical pathology table feature representation and table branch risk score through risk-related modeling. S3, perform tissue region detection, patch division and patch feature extraction on the WSI whole-slice pathological image, and perform region-level aggregation, slice-level aggregation and patient-level aggregation on the patch features in sequence to generate patient-level WSI pathological characterization and WSI branch risk score. S4, input the table branch risk score, the clinical pathology table feature representation, the patient-level WSI pathological characterization and the WSI branch risk score into the clinical-pathology anchor fusion module to generate Tabular+WSI anchor risk; S5. Based on the CT tumor mask, the tumor mask area of ​​each CT layer is statistically analyzed. The Top-K key layers whose tumor mask area meets the preset sorting conditions are selected. In each Top-K key layer, the lesion center is determined according to the tumor mask bounding box. The original lesion center cropped image, mask enhanced cropped image and tumor mask occupancy map are generated around the lesion center to obtain patient-level CT feature representation and CT branch risk score. S6, scale-align the Tabular+WSI anchor point risk and the CT branch risk score, calculate the residual correction term of the CT branch relative to the Tabular+WSI anchor point risk, and generate the final postoperative recurrence risk score based on the Tabular+WSI anchor point risk and the residual correction term. S7, output the patient's high-risk or low-risk stratification result based on the final postoperative recurrence risk score.

2. The method for generating a postoperative recurrence risk score for liver cancer based on stepwise pathology-imaging residual fusion according to claim 1, characterized in that, In step S1, the patient-level multimodal sample is represented as follows: ； in, Indicates the first Patient-level multimodal samples from one patient. Indicates the first Clinical pathology data for each patient Indicates the first WSI whole slide pathology images of one patient. Indicates the first CT image sequences of a patient, Indicates the first CT tumor mask sequences of one patient. This indicates the relapse-free survival time used during the training or validation phase. This indicates the relapse status used during the training or validation phase; and, patient-level data partitioning is used during the training and validation phases to ensure that multimodal data of the same patient do not appear simultaneously in the training and validation sets.

3. The method for generating a postoperative recurrence risk score for liver cancer based on stepwise pathology-imaging residual fusion according to claim 1, characterized in that, Step S2 includes: reading clinicopathological table variables including age, tumor size, alpha-fetoprotein (AFP), microvascular invasion (MVI), tumor stage, and inflammation-related indicators, and representing the clinicopathological table variables as follows: ； After imputing missing values ​​and standardizing the variables in the clinicopathology table, the clinicopathology table feature representation and table branch risk score are generated according to the following relationship: ； ； in, Indicates the number of clinicopathological variables. This represents the preprocessing and characterization functions for clinical pathology tables. This indicates the characteristics of a clinical pathology table. This represents the risk prediction function for clinical pathology tables. This indicates the risk score for each branch of the table.

4. The method for generating a postoperative recurrence risk score for liver cancer based on stepwise pathology-imaging residual fusion according to claim 1, characterized in that, Step S3 includes: performing tissue region detection on the WSI whole-section pathological image and removing blank background and invalid regions, and dividing the effective tissue region into a patch set: ； Feature extraction is performed on each patch to obtain patch-level features: ； The patch-level features are sequentially aggregated at the region level, slice level, and patient level to obtain the patient-level WSI pathological characterization: ； ； ； And based on the patient-level WSI pathological characteristics, the WSI branch risk score is calculated: ； in, Indicates the first A pathological image block, Indicates the number of valid patches. This represents the WSI patch feature extraction network. , and These represent region-level aggregation operations, slice-level aggregation operations, and patient-level aggregation operations, respectively. This represents the WSI pathology risk prediction function.

5. The method for generating a postoperative recurrence risk score for liver cancer based on stepwise pathology-imaging residual fusion according to claim 1, characterized in that, In step S4, the clinical-pathology anchor fusion module uses the clinical pathology table branch as the basic risk branch and the WSI pathology branch as the supplementary pathology information branch to enhance and correct the basic clinical risk, and generates the Tabular+WSI anchor risk according to the following relationship: ； in, This indicates the risk of Tabular+WSI anchor points. This indicates the risk score for each branch of the table. This indicates the WSI branch risk score. This indicates the supplementary risk correction item for the WSI branch relative to the table branch. Indicates the gating weight, and This represents the fusion coefficient.

6. The method for generating a postoperative recurrence risk score for liver cancer based on stepwise pathology-imaging residual fusion according to claim 1, characterized in that, Step S5 includes: representing the CT image sequence and the CT tumor mask sequence as follows: ； ； Statistics Tumor masking area on CT scan: ； Remove empty mask layers and sort the tumors by mask area from largest to smallest, selecting the largest area from the front. Each level serves as a key lesion level.

7. The method for generating a postoperative recurrence risk score for liver cancer based on stepwise pathology-imaging residual fusion according to claim 6, characterized in that, For each selected key lesion layer, the lesion center is determined based on the outer bounding box of the tumor mask: ； Set the center trimming length of the lesion: ； by Cut a square around the center and adjust the cutting area uniformly. Size, to obtain the original lesion center cropped image Masking Enhancement Image Cropping and tumor mask occupancy map ;in, , , , These represent the coordinate ranges of the bounding box of the tumor mask. Indicates the proportion of the surrounding context expansion of the lesion.

8. The method for generating a postoperative recurrence risk score for liver cancer based on stepwise pathology-imaging residual fusion according to claim 7, characterized in that, Steps S5 and S6 include: cropping the original lesion center image from multiple key lesion layers. The mask-enhanced cropped image and the tumor mask occupancy map The data is aggregated to form a patient-level CT feature representation, and a CT branch risk score is generated using a CT risk prediction function. ； ； Percentile rank normalization was performed on the Tabular+WSI anchor point risk and the CT branch risk scores to obtain: ； ； Calculate the CT residual correction term and the final postoperative recurrence risk score: ； ； in, This represents the CT crop feature aggregation function. This represents the CT branch risk prediction function. This represents the percentile rank normalization function fitted within the training fold. This represents the CT residual correction term. This represents the residual correction factor.

9. A system for generating a postoperative recurrence risk score for liver cancer based on hierarchical pathology-imaging residual fusion, characterized in that, include: The data acquisition module is used to acquire clinical pathology table data, WSI whole slide pathology images, CT serial slides, and CT tumor masks for the same patient. The patient-level multimodal data alignment module is used to perform patient-level consistency matching of the clinical pathology table data, the WSI whole-slice pathology images, the CT serial slices, and the CT tumor mask according to the patient ID; The clinical table feature processing module is used to perform missing value imputation, standardization, and risk-related modeling on the clinical pathology table data, and output clinical pathology table feature representation and table branch risk score. The WSI patient-level characterization module is used to perform tissue region detection, patch segmentation, patch feature extraction, region-level aggregation, slice-level aggregation and patient-level aggregation on the WSI whole-slice pathological images, and output patient-level WSI pathological characterization and WSI branch risk score. The CT lesion center cropping module is used to select the Top-K key layers based on the CT tumor mask, generate the original lesion center cropping image, the mask-enhanced cropping image and the tumor mask occupancy map around the lesion center, and output the patient-level CT feature representation and CT branch risk score. The clinical-pathological anchor fusion module is used to generate Tabular+WSI anchor risks based on the tabular branch risk scores and the WSI branch risk scores. The residual complementary fusion module is used to scale-align the Tabular+WSI anchor point risk and the CT branch risk score, calculate the CT residual correction term, and output the final postoperative recurrence risk score. The risk score output module is used to output high-risk or low-risk stratification results based on the final postoperative recurrence risk score.

10. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory, wherein the computer program, when executed by the processor, implements the method for generating a postoperative recurrence risk score for liver cancer based on hierarchical pathology-imaging residual fusion as described in any one of claims 1 to 8.

Images