Deep learning-based liver tumor post-ablation recurrence level prediction

Abstract

The invention provides a deep learning-based recurrence level prediction system after liver tumor ablation, and mainly solves the problems that after a doctor performs liver tumor ablation on a patient, whether recurrence occurs or not is difficult to predict, and the recurrence level evaluation is inaccurate. Firstly, the system performs feature extraction and analysis by using a 3D convolutional neural network by inputting a medical image after liver ablation and a given ablation range so as to judge whether a tumor relapses or not. If the prediction result is no relapse, ending the process; and if the prediction result is recurrence, further analyzing by using another 3D convolutional neural network model specially aiming at the recurrence level so as to determine the specific recurrence level (such as level 1, level 2 or level 3). According to the method based on deep learning, the accuracy of predicting the recurrence condition after liver tumor ablation can be improved, refined recurrence grading can be achieved, and scientific and reliable decision support is provided for clinicians.

Description

I. Technical Field

[0001] This invention belongs to the field of medical devices and can better assist doctors in preoperative planning by predicting the recurrence level after liver tumor ablation. II. Background Technology:

[0002] Localized progression (LTP) refers to the recurrence of liver tumors at the edge or within the ablation zone after ablation. It is primarily caused by insufficient ablation area, tumor biological characteristics, and technical limitations. Given the subjectivity and variability in determining the ablation extent based on physician experience, the difficulty in precise quantification, the limited capacity to handle complex cases, and the low efficiency of follow-up monitoring, introducing deep learning models to predict LTP recurrence and its severity is crucial. Deep learning models, through learning from extensive historical data, can automatically identify risk factors associated with LTP and provide more accurate recurrence predictions than human experts. Furthermore, this algorithmic analysis reduces human bias, ensuring that each case is evaluated according to the same high standards, thereby improving diagnostic consistency and reliability. Combined with imaging data, deep learning models can not only identify potential LTP signs earlier, helping physicians adjust treatment plans in a timely manner, but also tailor the most suitable ablation strategy for each patient, optimizing treatment outcomes and improving patient prognosis. Therefore, deep learning-based methods not only compensate for the shortcomings of traditional methods but also significantly improve the scientific rigor and effectiveness of liver cancer ablation therapy.

[0003] The three-dimensional visualization surgical system developed in the literature “Wang Yan, Dong Linan, Liu Fangyi, et al. Application value of three-dimensional visualization system in evaluating the efficacy of microwave ablation therapy for primary liver cancer [J]. Journal of PLA Medical College, 2022, 43(8): 840-844” can be used for preoperative planning to assist ablation therapy. It can also accurately and quantitatively evaluate the efficacy of tumor ablation through three-dimensional reconstruction and registration fusion.

[0004] The literature “Zhu X, Zhu J, Sun C, et al. Prediction of Local Tumor Progression After Thermal Ablation of Colorectal Cancer Liver Metastases Based on Magnetic Resonance Imaging Δ-Radiomics[J]. Journal of Computer Assisted Tomography, 2022:10.1097” constructed a clinical model based on clinical data and created two combined models, which combined clinical data with radiomics features at different stages, and used feature dimensionality reduction and machine learning methods to predict the progression of LTP.

[0005] The paper "Han K, Kim J H, Yang SG, et al. A single-center retrospective analysis of periprocedural variables affecting local tumor progression after radiofrequency ablation of colorectal cancer liver metastases[J]. Radiology, 2021, 298(1):212-218." used statistical analysis methods, especially the Cox proportional hazards regression model in survival analysis, to assess the impact of different factors on local tumor progression (LTP).

[0006] The methods employed in the aforementioned literature primarily rely on traditional statistical and machine learning techniques, which prove inadequate when dealing with complex data relationships. Specifically, in feature engineering, researchers need to manually extract and construct task-relevant features, which is not only time-consuming and labor-intensive but also highly dependent on domain knowledge and personal experience, making it difficult to comprehensively capture potential patterns in the data. Furthermore, because these methods cannot support end-to-end learning—that is, the entire process of directly mapping from raw input to final output—they have limitations in the expressive power and predictive accuracy of the models. III. Summary of the Invention

[0007] This invention proposes a deep learning-based system for predicting the recurrence level of liver tumors after ablation, aiming to address the challenge of accurately predicting recurrence and assessing the recurrence level after liver tumor ablation treatment. Traditional assessment methods rely heavily on physician experience and routine imaging examinations, which suffer from high subjectivity and insufficient accuracy, potentially leading to imprecise subsequent treatment plans and impacting patient prognosis. Therefore, this invention utilizes advanced deep learning technology, particularly 3D convolutional neural networks (3D CNNs), to extract and analyze features from post-ablation medical images, thereby achieving automated and high-precision prediction of liver tumor recurrence and providing further refined recurrence grading assessment.

[0008] The system's overall workflow comprises five main modules: data input, feature extraction and analysis, recurrence assessment, recurrence level evaluation, and result output. First, the system requires input of medical images of the liver after ablation and information about the given ablation range. This data forms the basis for subsequent analysis, and its accuracy and completeness directly impact the reliability of the prediction results. Next, the system extracts features from the input medical images using a 3D convolutional neural network, generating a 3D convolutional feature map rich in information. This process not only preserves the spatial structure information of the original data but also enhances the expressive power of the features, laying a solid foundation for subsequent recurrence assessment and level evaluation. Next, the system uses fully connected layers to further analyze the extracted features to determine whether the tumor has recurred. If the result indicates no recurrence, the process ends; if the result indicates recurrence, the system will further analyze using another 3D convolutional neural network model specifically designed for recurrence level analysis to determine the specific level of recurrence (e.g., level 1, 2, or 3). Finally, the system outputs the prediction results to clinicians, including whether recurrence has occurred and the specific level of recurrence. Based on this information and the patient's actual condition, clinicians can develop personalized treatment plans to improve treatment outcomes.

[0009] The technical advantages and innovations of this invention are reflected in several aspects. First, the system employs a 3D convolutional neural network for feature extraction and analysis, enabling it to comprehensively capture the spatial and temporal features in medical images, thus improving feature expressiveness and predictive accuracy. Compared to traditional statistical or machine learning methods, 3D CNNs have significant advantages in processing three-dimensional medical image data, better preserving the spatial structure information of the data. Second, the system can not only determine whether a tumor has recurred but also further assess the recurrence level. This refined grading method provides clinicians with more detailed reference information, helping to formulate more precise treatment plans. Different levels of recurrence correspond to different clinical significance and treatment strategies; refined grading helps improve treatment effectiveness and patient survival rates. IV. Description of the attached drawings

[0010] Figure 1 This is a flowchart illustrating the specific implementation of the present invention.

[0011] Figure 2 This is a flowchart of the algorithm of the present invention. V. Detailed Implementation Methods:

[0012] Step 1: Establish a liver tumor recurrence prediction model after ablation. This model uses medical images of the liver after ablation and a given ablation area as input data. It utilizes a 3D convolutional neural network for feature extraction and analysis to determine whether the liver tumor has recurred. For cases without recurrence, the result is output directly; for cases of recurrence, the model proceeds to the next stage. The specific steps are as follows:

[0013] The training set used in this study includes medical images of liver ablation annotated by professional physicians, along with corresponding binarized mask images of the ablation area. These images not only record in detail the microscopic pathological and geometric features of liver tumors after ablation, but also accurately label them as "recurring" or "non-recurring," ensuring the high quality and accuracy of the dataset. Through the meticulous annotation by professional physicians, this dataset provides a reliable learning foundation for the model, helping to improve the accuracy and reliability of recurrence prediction.

[0014] The designed model employs an innovative dual-encoder architecture specifically optimized for recurrence prediction after liver tumor ablation. This architecture comprises two main components: one encoder processes 3D medical images of the liver tumor after ablation to extract histopathologically relevant features; the other encoder focuses on processing a binarized mask image marked with the ablation region boundaries to extract geometric features of the ablation extent. The features extracted by these two encoders are then integrated through a feature fusion module. Finally, the input is fed into a fully connected layer for classification, outputting the recurrence probability.

[0015] During training, the dataset containing annotations from professional doctors was first preprocessed, and a combination of cross-entropy loss and Dice loss was used to address class imbalance. The Adam optimizer, combined with cosine annealing, was used to dynamically adjust the learning rate, and data augmentation techniques were applied to improve the model's generalization ability. Model validation employed five-fold cross-validation, with evaluation metrics including AUC and F1-Score. During inference, the post-ablation medical image of the liver and the mask image of the ablation area were standardized and normalized. Features were then extracted using a dual encoder, generated by a feature fusion module, and finally input into a fully connected layer to obtain a prediction of "recurrence" or "no recurrence." This entire process ensured efficiency and accuracy from data preparation to prediction output.

[0016] Step Two: For cases where recurrence is possible, predict the recurrence level. The specific procedure is as follows:

[0017] The training dataset consisted of 3D medical images of patients who had undergone liver tumor ablation, annotated by professional physicians, along with binarized mask images of the ablation areas, and three levels of recurrence labels. The third category of recurrence samples was the rarest due to its clinical rarity. All annotations were independently completed by senior radiologists and confirmed through expert consultation, ensuring high data quality and accurate classification, laying a solid foundation for building a high-precision recurrence level prediction model.

[0018] The first encoder receives a 3D medical image of a liver tumor after ablation as input. This image is processed through a series of 3D convolutional layers (Conv3D), normalization layers (BatchNorm3D), and max pooling layers (MaxPool3D) to generate a feature representation f1 = Conv3D. n (Image), where Conv3D i , represents the i-th 3D convolution operation, and n is the number of convolutional layers.

[0019] The second encoder processes the binarized mask image marked with the boundaries of the ablation region, extracting geometric features f2 = Conv3D' using the same convolutional structure. n (Mask). Next, in the feature fusion module, we concatenate the feature vectors output by the two encoders to form a comprehensive feature vector f. combined =[f1;f2].

[0020] Subsequently, the synthesized feature vector is further processed through several fully connected layers (FC), and finally mapped to three possible recurrence levels: y = Softmax(W2·ReLU(W1·f combined +b1)+b2). ​​Where W1 and W2 are the weight matrices of the first and second fully connected layers, respectively. b1 and b2 are the corresponding bias terms. y∈R 3 This represents the probability distribution for each category.

[0021] Regarding the training process, considering the extreme scarcity and significant clinical importance of the third type of samples, this step employs a combination of a phased classification training strategy and oversampling techniques to improve model performance: Phase 1 training: A binary classification model is trained using all samples, aiming to distinguish between "Category 1" and "Categories 2 and 3," thereby establishing preliminary risk identification capabilities. Phase 2 training: Training is conducted using only Category 2 and 3 samples, aiming to further refine the distinction between localized and widespread recurrence.

[0022] In order to alleviate the classification bias caused by the scarcity of third-class samples during training, a class-based weighted sampling strategy was introduced to improve the model's attention to minority class samples during training. Where N c w represents the number of samples in class C. c To correspond to the class weights, the class balance and generalization performance during model training are further enhanced.

[0023] Using the method described above, the probability of a sample being selected is automatically adjusted based on the category weights during training, allowing the model to "see" rare samples of the third category (widespread recurrence) more frequently during training. This approach effectively improves the model's ability to identify rare categories, and is particularly suitable for real-world scenarios with imbalanced sample distribution in medical image analysis.

[0024] Building upon the model architecture from Step 1, this step effectively combines three levels of labeled data, a phased training strategy, and oversampling techniques to achieve accurate grading and prediction of the degree of recurrence after liver tumor ablation.

Claims

1. A deep learning-based system for predicting the recurrence level of liver tumors after ablation, characterized in that... Includes the following steps: Step 1: Establish a liver tumor recurrence prediction model after ablation. This model uses medical images of the liver after ablation and a given ablation area as input data. It utilizes a 3D convolutional neural network for feature extraction and analysis to determine whether the liver tumor has recurred. For cases without recurrence, the result is output directly; for cases of recurrence, the process proceeds to the next stage. The specific steps are as follows: The training set used in this study includes medical images of liver ablation annotated by professional physicians, along with corresponding binarized mask images of the ablation area. These images not only record in detail the microscopic pathological and geometric features of liver tumors after ablation, but also accurately label them as "recurring" or "non-recurring," ensuring the high quality and accuracy of the dataset. Through the meticulous annotation by professional physicians, this dataset provides a reliable learning foundation for the model, helping to improve the accuracy and reliability of recurrence prediction. The designed model employs an innovative dual-encoder architecture specifically optimized for recurrence prediction after liver tumor ablation. This architecture comprises two main components: one encoder processes 3D medical images of the liver tumor after ablation to extract histopathologically relevant features; the other encoder focuses on processing a binarized mask image marked with the ablation region boundaries to extract geometric features of the ablation extent. The features extracted by these two encoders are then integrated through a feature fusion module. Finally, the input is fed into a fully connected layer for classification, outputting the recurrence probability. During training, the dataset containing annotations from professional doctors was first preprocessed, and a combination of cross-entropy loss and Dice loss was used to address class imbalance. The Adam optimizer, combined with cosine annealing, was used to dynamically adjust the learning rate, and data augmentation techniques were applied to improve the model's generalization ability. Model validation employed five-fold cross-validation, with evaluation metrics including AUC and F1-Score. During inference, the post-ablation medical image of the liver and the mask image of the ablation area were standardized and normalized. Features were then extracted using a dual encoder, generated by a feature fusion module, and finally input into a fully connected layer to obtain a "recurrence" or "no recurrence" prediction. This entire process ensured efficiency and accuracy from data preparation to prediction output. Step Two: For cases where recurrence is possible, predict the recurrence level. The specific procedure is as follows: The training dataset consisted of 3D medical images of patients who had undergone liver tumor ablation, annotated by professional physicians, along with binarized mask images of the ablation areas, and three levels of recurrence labels. The third category of recurrence samples was the rarest due to its clinical rarity. All annotations were independently completed by senior radiologists and confirmed through expert consultation, ensuring high data quality and accurate classification, laying a solid foundation for building a high-precision recurrence level prediction model. The first encoder receives a 3D medical image of a liver tumor after ablation as input. This image is processed through a series of 3D convolutional layers (Conv3D), normalization layers (BatchNorm3D), and max pooling layers (MaxPool3D) to generate a feature representation f1 = Conv3D. n (Image), where Conv3D i , represents the i-th 3D convolution operation, and n is the number of convolutional layers. The second encoder processes the binarized mask image marked with the boundaries of the ablation region, extracting geometric features f2 = Conv3D' using the same convolutional structure. n (Mask). Next, in the feature fusion module, we concatenate the feature vectors output by the two encoders to form a comprehensive feature vector f. combined =[f1;f2]. Subsequently, the synthesized feature vector is further processed through several fully connected layers (FC), and finally mapped to three possible recurrence levels: y = Softmax(W2·ReLU(W1·f combined +b1)+b2). ​​Where W1 and W2 are the weight matrices of the first and second fully connected layers, respectively. b1 and b2 are the corresponding bias terms. y∈R 3 This represents the probability distribution for each category. Regarding the training process, considering the extreme scarcity and significant clinical importance of the third type of samples, this step employs a combination of a phased classification training strategy and oversampling techniques to improve model performance: Phase 1 training: A binary classification model is trained using all samples, aiming to distinguish between "Category 1" and "Categories 2 and 3," thereby establishing preliminary risk identification capabilities. Phase 2 training: Training is conducted using only Category 2 and 3 samples, aiming to further refine the distinction between localized recurrence and widespread recurrence. In order to alleviate the classification bias caused by the scarcity of third-class samples during training, a class-based weighted sampling strategy was introduced to improve the model's attention to minority class samples during training. Where N c w represents the number of samples in the i-th class. c To correspond to the class weights, the class balance and generalization performance during model training are further enhanced. Using the method described above, the probability of a sample being selected is automatically adjusted based on the category weights during training, allowing the model to "see" rare samples of the third category (widespread recurrence) more frequently during training. This approach effectively improves the model's ability to identify rare categories, and is particularly suitable for real-world scenarios with imbalanced sample distribution in medical image analysis. Building upon the model architecture from Step 1, this step effectively combines three levels of labeled data, a phased training strategy, and oversampling techniques to achieve accurate grading and prediction of the degree of recurrence after liver tumor ablation.

Images