Method and system for predicting lymph node metastasis of esophageal squamous carcinoma in T1 stage based on deep learning

Abstract

The invention discloses a deep learning-based method and system for predicting lymph node metastasis of esophageal squamous carcinoma at T1 stage. The method specifically comprises the following steps: collecting WSI data of an esophageal carcinoma patient; classifying the collected data; dividing each piece of WSI data into a plurality of image blocks with the same size, and preprocessing the image blocks; constructing the image blocks of each piece of WSI data into a graph structure; inputting the graph structure into a neural network model, and training the neural network model; the trained neural network model is used for evaluating the lymph node metastasis probability of the esophageal squamous carcinoma in the T1 stage. According to the invention, the AUC in an internal test reaches 0.949, and the accuracy rate of external verification reaches 90.1%.

Description

Technical Field

[0001] This invention belongs to the field of medical artificial intelligence technology, and in particular relates to a method and system for predicting lymph node metastasis of T1 stage esophageal squamous cell carcinoma based on deep learning. Background Technology

[0002] Esophageal squamous cell carcinoma is a malignant tumor of the digestive tract, and lymph node metastasis is a key factor affecting its prognosis. For early-stage (T1) esophageal squamous cell carcinoma, endoscopic submucosal dissection (ESD) has become the standard treatment. However, if postoperative pathological examination reveals lymphovascular invasion (LVI+), current clinical guidelines usually recommend additional surgical esophagectomy and lymph node dissection. However, extensive clinical practice shows that most LVI+ patients who undergo additional surgery actually have negative lymph node pathology results. This means they suffer significant surgical trauma, complication risks, and a decreased quality of life without actually benefiting from the surgery. Therefore, there is an urgent clinical need for a tool that can accurately predict the lymph node metastasis status of individual patients preoperatively to avoid unnecessary overtreatment and achieve truly personalized precision medicine.

[0003] Existing prediction methods largely rely on simple combinations of clinicopathological features (such as depth of invasion and degree of differentiation), resulting in limited accuracy and reliability that fail to meet the high precision requirements of clinical decision-making. While whole-slide digital pathology images (WSIs) contain comprehensive information about the tumor microenvironment, the deep and complex morphological features they contain cannot be recognized by the human eye. Therefore, how to leverage artificial intelligence technology to deeply mine the predictive information in WSIs and construct high-precision LNM prediction models has become a pressing technical challenge. Summary of the Invention

[0004] Purpose of the invention: In order to solve the problems existing in the prior art, the present invention provides a method for predicting lymph node metastasis of T1 stage esophageal squamous cell carcinoma based on deep learning.

[0005] Technical solution: This invention provides a method for predicting lymph node metastasis in T1 stage esophageal squamous cell carcinoma based on deep learning, specifically as follows:

[0006] Collect WSI data from esophageal cancer patients;

[0007] The collected data is classified; each WSI data image is divided into several image blocks of the same size, and the image blocks are preprocessed;

[0008] Construct a graph structure from the image patches of each WSI data;

[0009] The graph structure is input into the neural network model to train the neural network model;

[0010] The trained model was used to assess the probability of lymph node metastasis in T1 stage esophageal squamous cell carcinoma.

[0011] Furthermore, the specific method for collecting WSI data from esophageal cancer patients is as follows: the patient's pathological specimen is fixed with 10% neutral buffered formalin, sections are prepared on the cut surface that exposes the deepest infiltrated area, and then hematoxylin-eosin staining is performed. The stained sections are then scanned using a digital slide scanner to obtain WSI data.

[0012] Furthermore, the preprocessing specifically involves: dividing the WSI into non-overlapping 224×224 pixel image blocks, removing blank areas and tumor-free areas, then randomly flipping and rotating the image blocks horizontally / vertically, using histogram matching based on the LAB color space to achieve color standardization, applying random brightness and contrast perturbations to simulate actual staining variations, and finally normalizing the spatial coordinates of the image blocks to the [0,1] interval.

[0013] Furthermore, the image patch of each WSI data is constructed into a graph structure as follows: a multi-scale sampling strategy is introduced into the ResNet50 network. By randomly sampling different pyramid levels in the preprocessed image patch, and then passing it through the ResNet50 network, the image features are obtained. The ResNet50 network removes the end classification layer. Finally, the image feature graph structure is constructed based on the k-nearest neighbor algorithm.

[0014] Furthermore, the ResNet50 network and neural network model are trained using a class-weighted cross-entropy loss function, and the Adam optimizer is used for optimization, with an early stopping mechanism introduced during training.

[0015] Furthermore, when evaluating the trained ResNet50 network and neural network model, core evaluation metrics are set, including AUC curve, sensitivity, specificity, positive predictive value, and negative predictive value. With the negative predictive value being greater than a preset threshold as a constraint, Youden's J statistic is maximized to obtain the optimal discrimination threshold.

[0016] Furthermore, the neural network model includes a graph neural network model and a global aggregation and classification module. The graph neural network model includes a first graph convolutional layer, a second graph convolutional layer, and a global average pooling layer connected in sequence. The global aggregation and classification module includes a fully connected classification head and a softmax activation function. The fully connected classification head is connected to the global average pooling layer.

[0017] Furthermore, the method also includes generating an attention heatmap aligned with the original WSI space using a node-level attention mechanism.

[0018] A deep learning-based system for predicting lymph node metastasis in T1 stage esophageal squamous cell carcinoma includes a WSI data generation module for generating WSI data from specimen scans.

[0019] The preprocessing module is used to divide each WSI data into several image blocks of the same size and to preprocess the image blocks.

[0020] The graph structure module is used to construct a graph structure from the image blocks of each WSI data;

[0021] A neural network model, based on a graph structure, outputs the probability of lymph node metastasis in stage T1 esophageal squamous cell carcinoma.

[0022] A computer device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the steps of the method for predicting lymph node metastasis of T1 stage esophageal squamous cell carcinoma based on deep learning.

[0023] Beneficial Effects: This invention utilizes graph neural networks (GNNs) to deeply mine complex histopathological patterns in WSI that are difficult for the human eye to recognize. In internal testing, the AUC reached 0.949, and the external validation accuracy reached 90.1%, with a particularly high negative predictive value (NPV) of 96.9%, meaning that the model's prediction of a negative result is highly likely to be accurate. This invention addresses the clinical decision-making dilemma of whether LVI+ patients after ESD require additional surgery, providing decision-makers with data support and potentially reducing overtreatment. It also maintained high performance on independent external validation sets of patients requiring additional surgery and follow-up after ESD, demonstrating its good potential for clinical generalization. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the deep learning-based lymph node metastasis prediction method provided in an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of the framework for processing WSI using a graph neural network (GNN) model in an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of an attention heatmap.

[0027] Figure 4 The figures provided in this embodiment of the invention are receiver operating characteristic (ROC) curves of the model on internal and external test sets, where Figure a is the receiver operating characteristic curve of the model on the internal test set and Figure b is the receiver operating characteristic curve of the model on the external test set.

[0028] Figure 5This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0030] like Figure 1 As shown, this embodiment provides a deep learning-based method for predicting lymph node metastasis in T1 stage esophageal squamous cell carcinoma. Specifically, it includes the following steps:

[0031] 1. Data Preparation This invention retrospectively collected WSI data from 374 esophageal cancer patients who underwent esophagectomy at Zhongda Hospital Affiliated to Southeast University between January 2019 and December 2024. All patients underwent systematic lymph node dissection and were clearly divided into a lymph node-positive group (LNM+, n=72) and a lymph node-negative group (LNM-, n=302) based on postoperative pathology reports. From the 374 eligible cases, 88 LNM-negative patients were randomly selected, ultimately including 160 patients (LNM+: 72, LNM−: 88) to mitigate bias caused by the low metastasis rate and enhance model generalization ability. The cohort was randomly divided into a training set (70%; n=112) and a test set (30%; n=48) in a 7:3 ratio. Surgical cases were used for model training and internal validation; ESD cases were used only for external validation to simulate real-world clinical scenarios after ESD. The independent validation cohort included ESD patients with histopathologically confirmed lymph node status: LNM positivity was confirmed by surgical specimens after ESD, and LNM negativity also included patients who were relapse-free for 3 years after ESD. This time cutoff was chosen based on established oncological evidence that almost all untreated LNM patients experience recurrence within 3 years. Therefore, 3-year recurrence-free survival (RFS) was used as a surrogate endpoint to validate true lymph node negativity, ensuring that clinically meaningful cohort stratification was used for validation purposes.

[0032] 2. Routine histological assessment All specimens were immediately fixed in 10% neutral buffered formalin upon acquisition. Sections were prepared on the cut surfaces that exposed the deepest infiltrative areas: ESD specimens were sectioned to a thickness of 2–3 mm, and surgical specimens from esophagectomy were sectioned to a thickness of 4–5 mm, followed by hematoxylin and eosin (H&E) staining. All specimens were classified as well-differentiated, moderately differentiated, or poorly differentiated according to the 2019 WHO classification of digestive system tumors. Submucosal invasion depth was measured vertically from the muscularis mucosae and classified as SM1 (≤200 μm) or SM2 (>200 μm). Lymphovascular invasion (LVI) was assessed by a combination of immunohistochemistry (D2-40) and histochemistry (Victoria blue staining). Tumor budding was defined as a cluster of isolated cancer cells (≤5 cells) at the invasive front, classified as BD1 (0–4 cells), BD2 (5–9 cells), or BD3 (≥10 cells) based on the number per 200x magnification field. According to our institutional standards, if the specimen after ESD shows any of the following characteristics, additional surgical intervention is recommended: (1) submucosal invasion depth >200 μm (SM2); (2) presence of LVI; (3) poorly differentiated pathological type; (4) positive vertical or horizontal resection margins; (5) tumor budding grade ≥ BD2. For surgically resected cases, the number of metastatic lymph nodes and the total number of lymph nodes dissected are extracted from the pathology report, and the number of lymph nodes collected is assessed to determine whether the staging is reliable according to the AJCC guidelines (≥15 lymph nodes dissected is considered reliable staging).

[0033] 3. Image preprocessing and feature extraction

[0034] H&E-stained sections from all tissue blocks of each patient were selected for further analysis. WSIs were acquired using a NanoZoomer (Hamamatsu Photonics) at 40x magnification. Tumor regions were labeled using QuPath by two experienced pathologists. All results were double-checked, and any discrepancies were discussed by an independent, blinded pathologist. The acquired WSIs were divided into non-overlapping 224×224 pixel image blocks; blank and cancer-free areas were removed. Each image block was assigned sections and labels according to the corresponding patient's LNM status; image blocks from cases without LNM were defined as LNM negative.

[0035] All WSIs underwent quality control, excluding sections without tumor tissue, and 659 high-quality WSIs were ultimately included for model development. Subsequently, each WSI was segmented into small image patches of 224x224 pixels at a magnification of 20x using open-source tools (such as OpenSlide).

[0036] Graph Construction and Model Training

[0037] To balance computational efficiency and tissue representativeness, a maximum of 1000 image patches were extracted from each WSI for subsequent analysis. For data augmentation, random horizontal / vertical flipping and 30° rotation were employed to improve robustness to orientation changes. A multi-scale sampling strategy (scaling ratio 0.5~1.5x) was introduced. To mitigate the impact of staining differences, histogram matching based on the LAB color space was used for color standardization, and random brightness and contrast perturbations (±20%) were applied to simulate actual staining variations. Scale invariance was further improved through random sampling at different pyramid levels. Feature extraction used a pre-trained ResNet50 architecture (with the end classification layer removed) to generate 2048-dimensional feature vectors, which were then reduced to 512 dimensions through fully connected layers. The spatial coordinates of the image patches were uniformly normalized to the [0,1] interval, and an adjacency graph was constructed based on the k-nearest neighbor algorithm (k=10) to characterize the spatial topological relationships between adjacent tissue regions. A nearest neighbor count of k=10 was validated in ablation experiments (comparing k=5, 10, 15, 20) to achieve the optimal balance between local context awareness and computational efficiency. A supervised learning approach was used to train a graph neural network (GNN) model (such as a Graph Attention Network, GAT). The model's input was the constructed graph data, and its output was the probability (a continuous value between 0 and 1) of lymph node metastasis in the patient corresponding to the WSI. The training process used a cross-entropy loss function, with Adam as the optimizer, iteratively optimizing the model parameters through backpropagation until the model loss function converged.

[0038] 4. Architecture of Graph Neural Networks

[0039] like Figure 2 As shown, the hierarchical graph neural network (GNN) model proposed in this embodiment is trained based on the graph structure constructed from WSI. First, each WSI image is segmented into non-overlapping 224×224 pixel image blocks. The feature vector of each image block is extracted using a pre-trained ResNet-50 backbone (initial dimension 2048), and then reduced to 512 dimensions using a linear compression layer. Subsequently, a graph structure is constructed based on the spatial adjacency relationships between image blocks, with each image block serving as a node in the graph, and adjacent image blocks connected by edges.

[0040] The graph neural network model employs two graph convolutional layers (GCNConv, 512 hidden units) to process the graph structure, using ReLU activation and Dropout to enhance generalization. Global average pooling aggregates all node features into a slice-level embedding representation. Finally, a fully connected classification head (512-dimensional input, 2-unit output) combined with a softmax activation function outputs the LNM's predicted probability. In this architecture, the GCN layer effectively captures the spatial structural patterns within the tumor microenvironment, while global pooling integrates local features into predictions at the entire WSI level. The entire model balances modeling relationships between local regions with extracting global semantic information.

[0041] 5. Supervised training

[0042] This model is trained using the PyTorch Lightning framework and employs a class-weighted cross-entropy loss function to mitigate the class imbalance problem. The optimization process uses the Adam optimizer with an initial learning rate of 1×10⁻⁻. 4 The learning rate was dynamically adjusted using the "ReduceLROnPlateau" scheduler (scaling factor = 0.1, patience = 5 rounds). An early stopping mechanism (patience = 4000 rounds) was introduced during training, using validation set accuracy as a monitoring metric to prevent overfitting. To improve training efficiency, 16-bit mixed-precision training was employed on an NVIDIA A100 GPU. In actual training, the early stopping condition was triggered when the validation loss showed no improvement for 50 consecutive rounds, thus ensuring model convergence while avoiding overfitting. The model was evaluated using five-fold cross-validation, and its performance was consistent across different data partitions (accuracy: 88.7% ± 1.1%, F1 score: 0.85 ± 0.03). The results of three independent trials under different random seeds also showed high agreement (accuracy: 89.2% ± 1.3%, F1 score: 0.87 ± 0.02), indicating that the model has low variance and good reproducibility. Furthermore, robustness testing of enhancement strategies (including rotation, flipping, and multi-scale sampling) showed that the model exhibited minimal performance fluctuations under different image transformations (accuracy variation <1.5%), indicating strong feature invariance. This multi-dimensional validation framework effectively ensured the statistical reliability of the model results, minimizing the risk of bias during clinical deployment.

[0043] 6. Post-training model evaluation

[0044] The area under the receiver operating characteristic (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were used as core performance metrics for the model. The optimal discrimination threshold was determined by maximizing the Youden's J statistic (J = sensitivity + specificity - 1), which aims to balance the model's ability to distinguish between positive and negative classes. In the threshold selection process, we introduced clinical constraints: optimizing the Youden's J index while ensuring NPV > 95% was a strategy based on a clinical risk trade-off: false negatives (i.e., missed diagnosis of LNM) may lead to undertreatment and therefore need to be strictly controlled; conversely, false positives (leading to unnecessary surgery) are within an acceptable clinical risk range. The model output probability was binarized using the finally determined threshold and compared with the pathological gold standard to comprehensively evaluate its discrimination performance and clinical applicability.

[0045] 7. Model interpretability and attention heatmap generation

[0046] To enhance the interpretability of the model, this study employs a node-level attention mechanism to analyze the hierarchical GNN, generating an attention heatmap aligned with the original WSI space. This attention heatmap is shown below. Figure 3 As shown, the core process involves segmenting a whole-slice image (WSI) into 224×224 pixel patches, then calculating the contribution weight (attention score) of each patch to the prediction of lymph node metastasis (LNM) using a model. Following a color rule of "warm colors, high weight; cool colors, low weight," the scores are mapped to the original WSI spatial coordinates to generate a visual heatmap. Each node in the heatmap corresponds to a histopathological image patch, and its attention weight, after normalization, is visualized as color intensity, directly reflecting the relative contribution of each image patch to the model's prediction results. After the heatmap is generated, it undergoes blind review by two physicians with over 10 years of experience in gastrointestinal pathology to confirm the consistency between the highlighted areas and high-risk pathological features of LNM.

[0047] In the heatmap, warm-colored areas primarily correspond to three types of high-risk LNM structures: the tumor invasion front (the boundary where tumor invades normal tissue), tumor sprouting foci (isolated clusters of ≤5 cancer cells at the invasion edge), and lymphovascular invasion areas (vascular structures containing tumor cells). Furthermore, the color intensity is positively correlated with LNM risk. In practical applications, this can be used to verify the rationality of model decisions (to see if the highlighted areas match actual metastatic features), assist pathologists in locating small high-risk lesions, and provide clinicians with intuitive evidence of "why the model determines the risk level," thereby improving the reliability of AI recommendations.

[0048] The visualization results consistently showed that high-attention areas were significantly concentrated in regions with clear histopathological significance, including the tumor-stromal junction, lymphocyte-rich areas, and vascular clusters—all established morphological features associated with LNM risk. All heatmaps were independently blinded and evaluated by two senior gastrointestinal pathologists (each with over 10 years of experience), confirming a high degree of consistency between the highlighted areas and known pathological risk regions, thus validating the effectiveness of the attention mechanism in identifying biologically relevant features.

[0049] 8. Model Validation like Figure 4 As shown in 'a', the model performed excellently on the internal test set, with an area under the receiver operating characteristic (AUC) of 0.949 (95% confidence interval: 0.912–0.986). To further validate its clinical applicability, an external validation set was constructed, including 35 patients after ESD: 18 patients underwent additional surgery due to LVI+ (4 LNM+, 14 LNM-), and 17 patients did not undergo surgery and were follow-up for 3 years without recurrence (considered LNM-). Figure 4 As shown in b, the model achieved an overall accuracy of 90.1% and a specificity of 91.4% on the validation set, successfully identifying 81.8% of positive cases. Crucially, its negative predictive value (NPV) reached 96.9%, demonstrating its significant value in screening low-risk patients and avoiding unnecessary surgeries.

[0050] 9. Clinical Applications Clinicians can digitize pathological slides from T1 stage esophageal squamous cell carcinoma patients diagnosed with LVI+ after ESD surgery into WSI and input them into this system. The system outputs a probability of lymph node metastasis risk. If the probability value is lower than a preset threshold (e.g., 0.1), it indicates an extremely low risk of LNM, and doctors can prioritize close follow-up rather than additional surgery; if the probability value is high, surgery is recommended. This system provides a powerful intelligent tool to assist clinical decision-making.

[0051] The hardware structure diagram of the electronic device provided in the embodiments of the present invention is as follows: Figure 5 As shown, the system includes an external input device, a computing server, and an input interaction device. The external input device is a whole-slice scanner used to scan specimens into WSI images. The computing server processes the WSI images using a deep learning model to generate the probability of lymph node metastasis in T1 stage esophageal squamous cell carcinoma. The input interaction device includes a display, an input device, and a cooling system. The display shows the heatmap and prediction results, and the input device is used to set the relevant parameters of the deep learning model.

[0052] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present invention will not describe the various possible combinations separately.

Claims

1. A method for predicting lymph node metastasis of T1 esophageal squamous cell carcinoma based on deep learning, characterized in that, Specifically: Collect WSI data from esophageal cancer patients; The collected data is classified; each WSI data image is divided into several image blocks of the same size, and the image blocks are preprocessed; Construct a graph structure from the image patches of each WSI data; The graph structure is input into the neural network model to train the neural network model; The trained model was used to assess the probability of lymph node metastasis in T1 stage esophageal squamous cell carcinoma. 2.The method of predicting lymph node metastasis of T1 esophageal squamous cell carcinoma based on deep learning according to claim 1, characterized in that, The specific process for collecting WSI data from esophageal cancer patients is as follows: the patient's pathological specimens are fixed with 10% neutral buffered formalin, sections are prepared on the cut surface that exposes the deepest infiltrated area, and then hematoxylin-eosin staining is performed. The stained sections are then scanned using a digital slide scanner to obtain WSI data. 3.The method of claim 1, wherein the method is characterized by, The preprocessing specifically involves: dividing the WSI into non-overlapping 224×224 pixel image blocks, removing blank areas and tumor-free areas, then randomly flipping and rotating the image blocks horizontally / vertically, using histogram matching based on the LAB color space to achieve color standardization, applying random brightness and contrast perturbations to simulate actual staining variations, and finally normalizing the spatial coordinates of the image blocks to the [0,1] interval. 4.The method of predicting lymph node metastasis of T1 esophageal squamous cell carcinoma based on deep learning according to claim 1, characterized in that, The specific steps for constructing a graph structure from each WSI data image patch are as follows: a multi-scale sampling strategy is introduced into the ResNet50 network. By randomly sampling different pyramid levels in the preprocessed image patch, and then passing the sample through the ResNet50 network to obtain image features, the ResNet50 network removes the end classification layer, and finally constructs the image feature graph structure based on the k-nearest neighbor algorithm.

5. The method for predicting lymph node metastasis of T1 esophageal squamous cell carcinoma based on deep learning according to claim 4, characterized in that, When training the ResNet50 network and neural network models, the class-weighted cross-entropy loss function is used, and the Adam optimizer is used in the optimization process. An early stopping mechanism is introduced during training.

6. The method for predicting lymph node metastasis of T1 esophageal squamous cell carcinoma based on deep learning according to claim 4, characterized in that, When evaluating the trained ResNet50 network and neural network model, core evaluation metrics are set, including AUC curve, sensitivity, specificity, positive predictive value, and negative predictive value. With the negative predictive value being greater than a preset threshold as a constraint, Youden's J statistic is maximized to obtain the optimal discrimination threshold.

7. The method for predicting lymph node metastasis of T1 esophageal squamous cell carcinoma based on deep learning according to claim 1, characterized in that, The neural network model includes a graph neural network model and a global aggregation and classification module. The graph neural network model includes a first graph convolutional layer, a second graph convolutional layer, and a global average pooling layer connected in sequence. The global aggregation and classification module includes a fully connected classification head and a softmax activation function. The fully connected classification head is connected to the global average pooling layer. 8.The method of predicting lymph node metastasis of T1 esophageal squamous cell carcinoma based on deep learning according to claim 1, characterized in that, The method also includes generating an attention heatmap aligned with the original WSI space using a node-level attention mechanism.

9. A deep learning-based system for predicting lymph node metastasis in T1 stage esophageal squamous cell carcinoma, characterized in that, Includes a WSI data generation module, used to scan specimens into WSI data; The preprocessing module is used to divide each WSI data into several image blocks of the same size and to preprocess the image blocks. The graph structure module is used to construct a graph structure from the image blocks of each WSI data; A neural network model, based on a graph structure, outputs the probability of lymph node metastasis in stage T1 esophageal squamous cell carcinoma.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for predicting lymph node metastasis of T1 stage esophageal squamous cell carcinoma based on deep learning as described in claim 1.

Images