Gastric cancer lymph node metastasis risk prediction method based on spatial perception multiple instance learning

Abstract

The present application relates to the technical field of computer vision and intelligent medical treatment, and particularly discloses a gastric cancer lymph node metastasis risk prediction method based on space perception multi-instance learning. After obtaining H&E and five kinds of immunohistochemical whole section images of a patient, the present application divides the images into image blocks and extracts features; a side graph relationship multi-instance learning predictor is independently constructed for each staining type, high contribution degree nodes and low contribution degree nodes are screened through self-attention scoring, a topological graph is constructed based on spatial physical coordinates, a learnable relationship embedding is introduced for iterative message passing and feature updating, and then the lymph node metastasis probabilities of each channel are output; six probabilities are evenly averaged through soft voting to form an AI immunoscore, and the metastasis risk is determined according to the ROC cutoff value. The present application replaces expensive multiplex immunohistochemistry with conventional single staining IHC, and realizes cross-center high-precision prediction through space graph reasoning and probability fusion cooperation.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and smart healthcare, and in particular to a method for predicting the risk of gastric cancer lymph node metastasis based on spatially perceptual multi-instance learning. Background Technology

[0002] Mixed-type early gastric cancer, due to the presence of both differentiated and undifferentiated histological components within the lesion, exhibits highly aggressive biological behavior and is accompanied by a significantly increased risk of lymph node metastasis. In clinical treatment decisions, accurately assessing the patient's lymph node metastasis (LNM) status is crucial for determining the feasibility of endoscopic minimally invasive resection and selecting an appropriate extent of radical surgery.

[0003] Currently, computer-aided pathological diagnosis mainly relies on digital whole-slice images (WSI) stained with hematoxylin and eosin (H&E). A breast cancer whole-slice image classification method, published in CN117237733A, combines self-supervised and weakly supervised learning. Based on a weakly supervised multi-instance learning framework, it crops whole-slice pathological images into image patch sets and uses deep neural networks to extract macroscopic morphological features such as cell morphology, tissue atypia, and glandular structure for tumor subtyping or prognostic risk assessment. However, when predicting processes involving complex microenvironment evolution, such as the risk of lymph node metastasis in mixed-type early gastric cancer, deep learning models based solely on H&E show significant limitations. Lymph node metastasis is controlled by the infiltration status and synergistic effects of specific immune cell subsets in the tumor immune microenvironment, such as cytotoxic T cells and macrophages. H&E staining only reflects the basic physical morphology of tissues and cannot specifically distinguish different immune cell subsets at the functional protein level, thus limiting the predictive specificity of the model at the molecular immunology level. Meanwhile, existing MIL frameworks typically treat image patches within a slice as an unordered set for feature aggregation, losing sight of the original physical spatial relative positions of the image patches. In reality, the spatial distribution of immune cells and tumor cells at the tissue invasion front is the core biological marker for assessing metastasis, and current technologies lack explicit network topology modeling for such complex spatial interactions.

[0004] To overcome the lack of specific immunological characteristics in H&E staining, clinicians sometimes attempt to use molecular pathological techniques such as multiplex immunofluorescence (IHC). However, these techniques either completely destroy the in-situ spatial structure of cells, or require cumbersome multiple rounds of antibody elution and staining imaging, and are highly dependent on extremely expensive specialized analytical equipment and high-cost reagents, making them difficult to popularize in routine clinical practice. While traditional single-index immunoscore methods have been applied, they mostly rely on manual or software-assisted inefficient counting of absolute cell numbers in specific areas. This not only suffers from significant subjective bias and low efficiency, but also fails to deeply analyze the complex structure of the tumor immune microenvironment at the multidimensional spatial network level. Therefore, how to automatically and quantitatively reconstruct the in-situ spatial topological connections between tumor cells and various immune cell populations using readily available multi-source pathological sections, while ensuring low cost and high clinical feasibility, and achieving high-precision prediction of metastasis risk, is a bottleneck problem that current digital pathology diagnostic technology urgently needs to solve. Summary of the Invention

[0005] This invention addresses the problems in the prior art, such as asymmetric representation of multimodal features, interference of slice alignment errors on spatial topology modeling, and difficulties in integrating heterogeneous multi-source quantitative indicators. It proposes a method for predicting the risk of lymph node metastasis in gastric cancer based on spatially perceptive multi-instance learning.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for predicting the risk of lymph node metastasis in gastric cancer based on spatially perceptive multi-instance learning, comprising the following steps: Obtain pathological whole slide images of the patient under test with multiple staining types, including H&E staining images and five immunohistochemical single staining images for CD3+, CD4+, CD8+, CD20+, and CD68+ immunomarkers; Each of the pathological whole slice images is segmented into non-overlapping image blocks, and feature extraction is performed on each image block to obtain the feature vector of each image block; For each staining type, a multi-instance learning predictor for edge graph relationships is constructed. The predictor calculates the contribution weight of each image patch to the prediction of lymph node metastasis through a self-attention scoring mechanism. Based on the contribution weight, the image patch is divided into a high-contribution node set and a low-contribution node set. Based on the high-contribution node set and the low-contribution node set, a node set of graph structure is constructed. Based on the absolute spatial physical coordinates of each node in the pathological whole slice image, spatial edges are constructed between nodes that meet a preset spatial distance threshold. Each spatial edge is assigned a learnable relation embedding vector to form a horizontal topology graph of the whole slice. The topology graph is iteratively message-passed and feature-updated in a graph neural network to output the lymph node metastasis risk probability corresponding to the staining type. The probability levels of lymph node metastasis risk corresponding to all staining types are fused to obtain the final lymph node metastasis risk assessment result for the patient under test.

[0007] Optionally, when performing feature extraction on each of the image patches, a CONCH encoder pre-trained on a large-scale pathological image dataset is used as the feature extraction backbone. All image patches corresponding to each WSI are input into the CONCH encoder one by one, and each image patch is transformed into a corresponding low-dimensional feature vector. , where P i For the i-th image patch, F i is the feature vector extracted from the i-th image patch, Encoder(·) is the CONCH encoder, and WSI is the full slice image.

[0008] Optionally, constructing a multi-instance learning predictor for edge graph relationships for each coloring type includes: The feature vectors of each image patch are dimensionality-reduced by using a linear projection layer to obtain the dimensionality-reduced feature representation z. i Specifically , Among them W p Let b be the projection matrix. p For bias vectors, , d is the dimension of the low-dimensional feature space, F i The feature vector extracted for the i-th image patch; The dimensionality-reduced feature representation is input into the dual-path linear layer and the self-attention scoring module to calculate the dynamic self-attention score for each image patch. The specific formula is as follows: , Among them, z i Let represent the dimensionality reduction feature representation of the i-th image patch, U and V be the weight matrices of the two linear layers respectively, w be the attention projection vector, ⊙ represent the Hadamard product, and sigm(·) be the gated activation function.

[0009] Optionally, the node set for constructing the graph structure based on the high-contribution node set and the low-contribution node set includes: The dynamic self-attention scores of each image patch are sorted in descending order, and the image patches with a contribution score higher than the first threshold are selected as the high contribution node set N. H Image patches with a contribution value lower than the second threshold are selected as the low contribution node set N. L Construct a graph structure with a node set V. .

[0010] Optionally, based on the spatial physical coordinates of each node in the pathological whole-slice image, spatial edges are constructed between nodes that meet a preset spatial distance threshold, including: Let node v be a record. i The absolute spatial physical coordinates in the pathological whole-section image are p i =(x i, y i When two nodes v i With v j When the Euclidean distance is less than a preset absolute spatial physical distance threshold, at node v i With v j Constructing spatial edges e ij =(v i, v j )∈E, thus obtaining the full slice horizontal topology graph G=(V,E).

[0011] Optionally, iterative message passing and feature updating are performed on the topology graph in the graph neural network, including: For each edge e in the graph structure ij =(v i ,v j )∈E, assign a learnable relation embedding vector r ij (0) ∈R dr ; During the t-th iteration of message passing, the message sent by neighboring node j to center node i is calculated according to the following formula: , Among them, MLP m (·) represents a multilayer perceptron used for message extraction, and || represents the concatenation operation of feature vectors; The aggregated message volume of node i is calculated according to the following formula, i.e. , Where N(i) represents the set of spatial neighborhood nodes of node i in the graph structure, a j Give an attention score to node j.

[0012] Optionally, performing iterative message passing and feature updating on the topology graph in the graph neural network further includes: Update the node's characteristic state z according to the following formula. i (t+1) ,Right now ; Update the relational embedding state r according to the following formula: ij (t+1) ,Right now , Among them, MLP u (·) and MLP r (·) represents the parameter update of the multilayer perceptron for the node state and the relationship embedding state, respectively.

[0013] Optionally, the output of the lymph node metastasis risk probability corresponding to this staining type includes: After T iterations of message passing, the graph features of the entire slice are integrated into a global feature vector Z through global pooling operations in the readout layer. global The expression is ; The global feature vector is input into a linear classifier, and the lymph node metastasis risk probability y corresponding to this staining type is calculated using an activation function. * The expression is , Among them, W o With b o These are the weight matrix and bias parameters of the output linear classification layer, respectively.

[0014] Optionally, the step of performing probability-level fusion of the lymph node metastasis risk probabilities corresponding to all staining types to obtain the final lymph node metastasis risk assessment result for the patient under test includes: The AI ​​immune score (AIS) is calculated using the following formula: , Where M is the number of staining types, P m (Y=1|X) m ) represents the positive confidence probability value of lymph node metastasis output by the multi-instance learning predictor of the side graph relationship corresponding to the m-th staining type; The cutoff value is determined based on the maximum point of the Youden index on the receiver operating characteristic curve. When the AIS is greater than the cutoff value, the patient is determined to be positive for lymph node metastasis; otherwise, the patient is determined to be negative for lymph node metastasis.

[0015] Compared with the prior art, the present invention includes at least one of the following beneficial technical effects: First, this invention treats each staining as an independent channel, utilizes a side graph relation multi-instance learning predictor to independently extract the unique morphological-spatial topological features of the channel and output LNM probability. Finally, at the decision level (patient level), the independent prediction results of the six channels are fused through probability-level soft voting (uniform averaging) to form the final AIS score. This avoids the dilemma of misalignment caused by differences in physical properties of different modal images. The biological information of each channel (macromorphology of H&E, specific immune subpopulations of each IHC) is fully explored in its own space, and then complementary fusion is performed on a unified risk probability scale to achieve effective integration of multi-source heterogeneous information. Second, this invention abandons cross-slice registration, completely avoiding the cumulative error caused by physical alignment between slices. In the independent physical spatial coordinate system of each slice, spatial edges are constructed based on the real spatial physical coordinates of the image blocks through a preset distance threshold, forming a unique full-slice horizontal topology map of the slice. This topology map only depends on the local spatial proximity relationship within a single slice and does not depend on any cross-slice information. It fundamentally bypasses the physical bottleneck that multiple consecutive slices cannot be accurately registered due to differences in slide preparation processes. The topology network reconstructed in the coordinate system of a single slice accurately reflects the real in-situ spatial arrangement of tumor cells, immune cells and stroma on the slice, ensuring the biological effectiveness of spatial interaction features (such as tumor-immune boundaries and invasion fronts). Third, this invention employs decision-level fusion rather than feature-level splicing. Each channel first independently completes a full spatial graph inference, outputting a quantitative indicator with unified biological meaning—LNM risk probability. This probability encodes the unique spatial topological interaction information of that channel. Finally, at the patient level, the LNM probabilities of the six channels are uniformly averaged. The outputs of all channels are probability values ​​of 0 to 1, which are on the same numerical scale. This avoids the problem of large values ​​being swallowed up by small values ​​caused by different dimensions and feature distributions in feature-level splicing. The prediction results of each channel have directly comparable and average mathematical meaning on the risk probability scale, and the fusion results are robust. Fourth, this invention utilizes artificial intelligence to perform deep learning on whole-slice images and multi-source immune marker images of pathologically diagnosed mixed early gastric cancer. The trained deep learning model can automatically and systematically complete the in-depth analysis of multi-source pathological morphological features and spatial immune microenvironment of mixed early gastric cancer, and output high-precision lymph node metastasis risk assessment results. This provides a new technical path to overcome the dilemma of difficulty in taking into account specific immune indicators and spatial topological relationships in conventional pathological assessment. Fifth, this invention develops a gastric cancer metastasis risk prediction model that integrates morphology and immune microenvironment to assist clinicians in making high-precision and standardized quantitative assessments of the risk of lymph node metastasis in mixed early gastric cancer. This model helps to identify patients with high metastasis risk who require additional radical surgery, while freeing low-risk patients from unnecessary additional surgery, thereby improving diagnostic efficiency and effectively reducing the occurrence of over-treatment and secondary surgical complications. Compared with human pathologists, the advantages of artificial intelligence technology with graph neural networks as the core in processing multi-source digital pathological whole-slice images are mainly reflected in the following aspects: (1) Advantages of global integration and correlation of multi-source and multi-pathway image features: It is difficult for human pathologists to intuitively and quantitatively establish multi-dimensional spatial correlations between macroscopic tumor morphology and microscopic multiple immune cell populations in multiple pathological slices, while the algorithm can efficiently aggregate these multi-modal information across slices to achieve a more comprehensive microenvironment profile. (2) Objective quantification capability at the physical topology level: This model utilizes multi-instance learning of edge graph relationships to transform abstract and heterogeneous immune infiltration patterns into a precise physical topology network composed of nodes and learnable relationship embeddings. It can quantify and extract sub-visual features such as intercellular spatial relative distances and interaction networks that cannot be precisely measured by the human eye. (3) High consistency and robust risk output: It has demonstrated robust and highly consistent risk discrimination capability in multicenter validation sets. (4) Significant cost-effectiveness and clinical applicability: It avoids expensive multiplex immunohistochemistry or spatial transcriptome sequencing, which have high technical barriers and are extremely difficult to promote in routine clinical practice. It only uses single-stained IHC slides, which are commonly available in primary hospitals, to obtain spatial immune heterogeneity assessment indicators with the same effect through digital algorithm fusion, demonstrating clinical applicability and promotion potential. Attached Figure Description

[0016] Figure 1 This diagram illustrates the steps of a method for predicting the risk of lymph node metastasis in gastric cancer based on spatially perceptive multi-instance learning. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, this embodiment provides a method for predicting the risk of lymph node metastasis in gastric cancer based on spatially aware multi-instance learning, including the following steps: Obtain pathological whole slide images of the patient under test with multiple staining types, including H&E staining images and five immunohistochemical single staining images for CD3+, CD4+, CD8+, CD20+, and CD68+ immunomarkers; Each of the pathological whole slice images is segmented into non-overlapping image blocks, and feature extraction is performed on each image block to obtain the feature vector of each image block; For each staining type, a multi-instance learning predictor for edge graph relationships is constructed. The predictor calculates the contribution weight of each image patch to the prediction of lymph node metastasis through a self-attention scoring mechanism. Based on the contribution weight, the image patch is divided into a high-contribution node set and a low-contribution node set. Based on the high-contribution node set and the low-contribution node set, a node set of graph structure is constructed. Based on the absolute spatial physical coordinates of each node in the pathological whole slice image, spatial edges are constructed between nodes that meet a preset spatial distance threshold. Each spatial edge is assigned a learnable relation embedding vector to form a horizontal topology graph of the whole slice. The topology graph is iteratively message-passed and feature-updated in a graph neural network to output the lymph node metastasis risk probability corresponding to the staining type. The probability levels of lymph node metastasis risk corresponding to all staining types are fused to obtain the final lymph node metastasis risk assessment result for the patient under test.

[0019] Specifically, after confirmation by senior gastrointestinal pathologists, the most representative tissue blocks were selected for serial sectioning for each patient. The sections were stained with Hematoxylin and eosin (H&E) using the Dako fully automated staining platform, as well as immunohistochemical staining targeting CD3+, CD4+, CD8+, CD20+, and CD68+ cells. CD3+ labeled T cells, CD4+ labeled helper T cells, and CD8+ labeled cytotoxic T cells; CD68+ served as a marker for monocytes / macrophages, while CD20+ was used to detect B cells. The stained sections were then digitally scanned using a Nanozoomer XP series and Leica GT450 scanner to obtain multi-source digital whole-slide images (WSI). Subsequently, the WSI images underwent quality review, discarding images with blurriness, tissue fading, and abnormal staining. At 20x magnification, blank background areas and tissue-free areas were first removed from the WSI, retaining only areas containing valid tissue information. The tissue area was then divided into non-overlapping image patches, each uniformly sized at 224×224 pixels.

[0020] Furthermore, when extracting features from each of the image patches, a CONCH encoder pre-trained on a large-scale pathological image dataset is used as the feature extraction backbone. All image patches corresponding to each WSI are input into the CONCH encoder one by one, and each image patch is transformed into a corresponding low-dimensional feature vector. , where P i For the i-th image patch, F i is the feature vector extracted from the i-th image patch, Encoder(·) is the CONCH encoder, and WSI is the full slice image.

[0021] Specifically, after each image patch is extracted by the CONCH encoder, the output is a 512-dimensional semantic feature vector, denoted as: D=512. The aforementioned CONCH is a pathological model that has been pre-trained using self-supervised contrastive learning on massive amounts of pathological image-text paired data, enabling it to capture high-dimensional and robust histological semantic feature representations.

[0022] Furthermore, the construction of a multi-instance learning predictor for edge graph relationships for each coloring type includes: The feature vectors of each image patch are dimensionality-reduced by using a linear projection layer to obtain the dimensionality-reduced feature representation z. i Specifically , Among them W p Let b be the projection matrix. p For bias vectors, , d is the dimension of the low-dimensional feature space, F i The feature vector extracted for the i-th image patch; The dimensionality-reduced feature representation is input into the dual-path linear layer and the self-attention scoring module to calculate the dynamic self-attention score for each image patch. The specific formula is as follows: , Among them, z i Let represent the dimensionality reduction feature representation of the i-th image patch, U and V be the weight matrices of the two linear layers respectively, w be the attention projection vector, ⊙ represent the Hadamard product, and sigm(·) be the gated activation function.

[0023] Furthermore, the node set for constructing the graph structure based on the high-contribution node set and the low-contribution node set includes: The dynamic self-attention scores of each image patch are sorted in descending order, and the image patches with a contribution score higher than the first threshold are selected as the high contribution node set N. H Image patches with a contribution value lower than the second threshold are selected as the low contribution node set N. L Construct a graph structure with a node set V. .

[0024] Furthermore, based on the spatial physical coordinates of each node in the pathological whole-slice image, spatial edges are constructed between nodes that meet a preset spatial distance threshold, including: Let node v be a record. i The absolute spatial physical coordinates in the pathological whole-section image are p i =(x i, y i When two nodes v i With v j When the Euclidean distance is less than a preset absolute spatial physical distance threshold, at node v i With v j Constructing spatial edges e ij =(v i, v j )∈E, thus obtaining the full slice horizontal topology graph G=(V,E).

[0025] Furthermore, iterative message passing and feature updating are performed on the topology graph in the graph neural network, including: For each edge e in the graph structure ij =(v i ,v j )∈E, assign a learnable relation embedding vector r ij (0) ∈R dr ; During the t-th iteration of message passing, the message sent by neighboring node j to center node i is calculated according to the following formula: , Among them, MLP m (·) represents a multilayer perceptron used for message extraction, and || represents the concatenation operation of feature vectors; The aggregated message volume of node i is calculated according to the following formula, i.e. , Where N(i) represents the set of spatial neighborhood nodes of node i in the graph structure, a j Give an attention score to node j.

[0026] Furthermore, the iterative message passing and feature updating of the topology graph in the graph neural network also includes: Update the node's characteristic state z according to the following formula. i (t+1) ,Right now ; Update the relational embedding state r according to the following formula: ij (t+1),Right now , Among them, MLP u (·) and MLP r (·) represents the parameter update of the multilayer perceptron for the node state and the relationship embedding state, respectively.

[0027] Furthermore, the output of the lymph node metastasis risk probability corresponding to this staining type includes: After T iterations of message passing, the graph features of the entire slice are integrated into a global feature vector Z through global pooling operations in the readout layer. global The expression is ; The global feature vector is input into a linear classifier, and the lymph node metastasis risk probability y corresponding to this staining type is calculated using an activation function. * The expression is , Among them, W o With b o These are the weight matrix and bias parameters of the output linear classification layer, respectively.

[0028] Specifically, the construction of a deep learning model for predicting lymph node metastasis in mixed-type early gastric cancer includes: A dataset was established from 410 patients with T1 stage gastric cancer from three centers: the First Affiliated Hospital of Anhui Medical University (FAHAMU), Nanjing Drum Tower Hospital (NJDTH), and Zhejiang Provincial People's Hospital (ZJPPH). Informed consent was obtained from all patients for this dataset. 410 H&E-stained whole-slice images (WSI) and 2050 IHC images were collected from the patients. CD3 labeled T cells, CD4 labeled helper T cells, and CD8 labeled cytotoxic T cells; CD68 was used as a marker for monocytes and macrophages, while CD20 was used to detect B cells. All slides were fully reviewed and digitized using a Nanozoomer XP or Leica GT450 fully automated digital scanner at 20x magnification. To ensure dataset quality, images with blurred, faded, or abnormally stained images underwent quality control screening. 35 cases were excluded from the Nanjing Drum Tower Hospital cohort, 11 cases from the Zhejiang Provincial People's Hospital cohort, and 5 cases from the First Affiliated Hospital of Anhui Medical University cohort. The resulting high-quality digitized whole-slice pathology images constituted a standard dataset for mixed-type early gastric cancer images. The image annotation work in this embodiment was completed collaboratively by senior gastrointestinal pathologists. Based on the postoperative regional lymph node biopsy report, the pathological sections were determined and labeled as "with lymph node metastasis (LNM positive)" and "without lymph node metastasis (LNM negative)".

[0029] To ensure the robustness of deep learning model training and the objectivity of evaluation, this embodiment allocates the dataset scientifically and reasonably. Digital pathological images from Nanjing Drum Tower Hospital are used as model building and internal validation data. Of these, 374 sample images are used as the training set for model building, and three-fold cross-validation is performed to select the optimal hyperparameters; 106 sample images are used as the internal test set for preliminary evaluation of model performance. Simultaneously, to test the model's independent diagnostic and generalization capabilities in real-world applications, 114 images from the First Affiliated Hospital of Anhui Medical University and 76 images from Zhejiang Provincial People's Hospital are used as completely independent external validation datasets.

[0030] Image preprocessing and data augmentation are crucial. Due to the extremely large size of WSI images, the model requires block preprocessing before feature extraction. Specifically, the algorithm segments the digital whole-slice image into non-overlapping pathological image blocks of 224×224 pixels at a magnification of 20x, removing blank backgrounds and tissue-free areas. To expand the training dataset and prevent overfitting, this embodiment performs extensive data augmentation operations on the image blocks during the training phase, including random transformations of hue, saturation, and brightness within a 50% range, and random horizontal, vertical, and multi-angle rotations of the image blocks. This improves the model's robustness to different staining batches and scanning effects.

[0031] The deep learning model is built, and the model construction mainly consists of four parts: First, a large-scale pre-trained model (CONCH encoder) on massive pathological images is used as a feature extractor to transform each input image patch into a 512-dimensional semantic feature vector. Second, a self-attention gating mechanism is introduced to dynamically assign weights based on the contribution of each image patch to LNM prediction, thereby suppressing stromal background noise and automatically focusing on the tumor core region. Third, an explicit graph topology network is constructed. Based on the descending order of attention scores, high-contribution image patches representing tumor nests are selected as high-attention nodes, and low-contribution image patches representing the stroma and background are selected as low-attention nodes, constructing a dual-class node embedding topology graph structure. Then, based on the spatial physical coordinates of each node in the slice, spatial connection edges are constructed between neighboring nodes, and each edge is assigned a learnable relation embedding, which is used in the graph neural network layer to iteratively pass messages, deeply aggregate the interaction features of nodes and edges, and accurately capture long-distance spatial contextual dependencies. The fourth step involves global pooling of the readout layer to integrate the deep structured graph features of the entire slice into a single global feature vector. This vector is then used by a classification output unit to obtain the LNM metastasis risk probability for each stained slice. Finally, for the same patient, the probabilities output from the spatial graph inference of the six channels (H&E, CD3, CD4, CD8, CD20, and CD68) are uniformly averaged to obtain the final AIS. Thus, the input to probability fusion is not the original image features or simple cell counts, but rather channel-level prediction results that have encoded the spatial topological interaction information of each stained channel. Spatial graph inference and multi-stained probability fusion form a synergistic relationship with each other in the same patient-level LNM decision-making process.

[0032] Model validation and results: In this embodiment, the predictive performance of AIS, H&E single-stain model, CD3, CD4, CD8, CD20 and CD68 single-stain model were compared in the internal test set and two independent external validation queues to verify the gain of multi-stain probability fusion relative to single-channel prediction. AIS score achieved extremely high prediction accuracy for MTEGC lymph node metastasis.

[0033] Table 1: AUC values ​​of LNM predicted by the model in three independent cohorts .

[0034] Table 2: Average prediction performance of the model across the three cohorts .

[0035] Furthermore, to verify the model's practical application potential, this embodiment compares the AIS score proposed in this invention with a traditional clinical logistic regression prediction model. This clinical logistic regression model is constructed using common clinicopathological risk factors such as tumor size, vascular invasion, and nerve invasion as core indicators. The comparative analysis results show that the AUC of the AIS score reaches 0.926, significantly better than the 0.768 of the traditional clinical model (P<0.001).

[0036] Multivariate logistic regression analysis of lymph node metastasis in mixed early gastric cancer showed that, after adjusting for confounding factors such as age, sex, tumor size, and depth of invasion, the AIS score output by this model was the only significant independent predictor (OR=30.21, 95%CI: 10.65-85.73, P<0.001).

[0037] Table 3: Univariate and Multivariate Logistic Regression Analysis Tables of Risk Factors for Lymph Node Metastasis .

[0038] Furthermore, the step of performing probability-level fusion of the lymph node metastasis risk probabilities corresponding to all staining types to obtain the final lymph node metastasis risk assessment result for the patient under test includes: The AI ​​immune score (AIS) is calculated using the following formula: , Where M is the number of staining types, P m (Y=1|X) m ) represents the positive confidence probability value of lymph node metastasis output by the multi-instance learning predictor of the side graph relationship corresponding to the m-th staining type; The cutoff value is determined based on the maximum point of the Youden index on the receiver operating characteristic curve. When the AIS is greater than the cutoff value, the patient is determined to be positive for lymph node metastasis; otherwise, the patient is determined to be negative for lymph node metastasis.

[0039] Specifically, for a patient clinically diagnosed with mixed-type early gastric cancer, serial sections of the surgical resection specimen are first obtained and subjected to H&E staining and immunohistochemical staining for CD3, CD4, CD8, CD20, and CD68. The six stained sections are digitized using a fully automated scanner and then input into the deep learning model trained according to this invention. The model performs feature extraction, graph topology construction, and spatial message passing on the image patches of each stained section, outputting the LNM risk probability for each of the six channels. The AIS score for the patient is then calculated through soft voting fusion. When the AIS score is higher than a preset cutoff value, the model outputs a prediction of "high risk of lymph node metastasis," suggesting that clinicians should consider additional radical surgery and lymph node dissection. Conversely, when the AIS score is lower than the cutoff value, the model outputs a prediction of "low risk of lymph node metastasis," supporting clinicians in choosing endoscopic follow-up observation or restrictive surgical options.

[0040] In addition, the model can also output a spatial attention heatmap at the whole slice level, which intuitively displays the tissue regions that contribute the most to the prediction results, providing clinicians with a visual basis for decision-making and enhancing the interpretability of the model's prediction results.

[0041] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for predicting the risk of lymph node metastasis in gastric cancer based on spatially perceptive multi-instance learning, characterized in that, Includes the following steps: Obtain pathological whole slide images of the patient under test with multiple staining types, including H&E staining images and five immunohistochemical single staining images for CD3+, CD4+, CD8+, CD20+, and CD68+ immunomarkers; Each of the pathological whole slice images is segmented into non-overlapping image blocks, and feature extraction is performed on each image block to obtain the feature vector of each image block; For each staining type, a multi-instance learning predictor for edge graph relationships is constructed. The predictor calculates the contribution weight of each image patch to the prediction of lymph node metastasis through a self-attention scoring mechanism. Based on the contribution weight, the image patch is divided into a high-contribution node set and a low-contribution node set. Based on the high-contribution node set and the low-contribution node set, a node set of graph structure is constructed. Based on the absolute spatial physical coordinates of each node in the pathological whole slice image, spatial edges are constructed between nodes that meet a preset spatial distance threshold. Each spatial edge is assigned a learnable relation embedding vector to form a horizontal topology graph of the whole slice. The topology graph is iteratively message-passed and feature-updated in a graph neural network to output the lymph node metastasis risk probability corresponding to the staining type. The probability levels of lymph node metastasis risk corresponding to all staining types are fused to obtain the final lymph node metastasis risk assessment result for the patient under test.

2. The method for predicting the risk of gastric cancer lymph node metastasis based on spatially aware multi-instance learning according to claim 1, characterized in that, When performing feature extraction on each of the aforementioned image patches, the CONCH encoder, pre-trained on a large-scale pathological image dataset, is used as the backbone for feature extraction. All image patches corresponding to each WSI are input into the CONCH encoder one by one, and each image patch is transformed into a corresponding low-dimensional feature vector. , where P i For the i-th image patch, F i is the feature vector extracted from the i-th image patch, Encoder(·) is the CONCH encoder, and WSI is the full slice image.

3. The method for predicting the risk of gastric cancer lymph node metastasis based on spatially aware multi-instance learning according to claim 1, characterized in that, The construction of a multi-instance learning predictor for side graph relationships for each coloring type includes: The feature vectors of each image patch are dimensionality-reduced by using a linear projection layer to obtain the dimensionality-reduced feature representation z. i Specifically ， Among them W p Let b be the projection matrix. p For bias vectors, , d is the dimension of the low-dimensional feature space, F i The feature vector extracted from the i-th image patch; The dimensionality-reduced feature representation is input into the dual-path linear layer and the self-attention scoring module to calculate the dynamic self-attention score for each image patch. The specific formula is as follows: ， Among them, z i Let represent the dimensionality reduction feature representation of the i-th image patch, U and V be the weight matrices of the two linear layers respectively, w be the attention projection vector, ⊙ represent the Hadamard product, and sigm(·) be the gated activation function.

4. The method for predicting the risk of gastric cancer lymph node metastasis based on spatially perceptive multi-instance learning according to claim 3, characterized in that, The node set that constructs the graph structure based on the high-contribution node set and the low-contribution node set includes: The dynamic self-attention scores of each image patch are sorted in descending order, and the image patches with a contribution score higher than the first threshold are selected as the high contribution node set N. H Image patches with a contribution value lower than the second threshold are selected as the low contribution node set N. L Construct a graph structure with a node set V. .

5. The method for predicting the risk of gastric cancer lymph node metastasis based on spatially aware multi-instance learning according to claim 4, characterized in that, Based on the absolute spatial physical coordinates of each node in the pathological whole-slice image, spatial edges are constructed between nodes that meet a preset spatial distance threshold, including: Let node v be a record. i The absolute spatial physical coordinates in the pathological whole-section image are p i =(x i, y i When two nodes v i With v j When the Euclidean distance is less than a preset absolute spatial physical distance threshold, at node v i With v j Constructing spatial edges e ij =(v i, v j )∈E, thus obtaining the full slice horizontal topology graph G=(V,E).

6. The method for predicting the risk of gastric cancer lymph node metastasis based on spatially aware multi-instance learning according to claim 5, characterized in that, Iterative message passing and feature updating of the topology graph in the graph neural network includes: For each edge e in the graph structure ij =(v i ,v j )∈E, assign a learnable relation embedding vector r ij (0) ∈R dr ; During the t-th iteration of message passing, the message sent by neighboring node j to center node i is calculated according to the following formula: ， Among them, MLP m (·) represents a multilayer perceptron used for message extraction, and || represents the concatenation operation of feature vectors; The aggregated message volume of node i is calculated according to the following formula, i.e. ， Where N(i) represents the set of spatial neighborhood nodes of node i in the graph structure, a j Give an attention score to node j.

7. The method for predicting the risk of gastric cancer lymph node metastasis based on spatially aware multi-instance learning according to claim 6, characterized in that, In a graph neural network, iterative message passing and feature updating of the topology graph also include: Update the node's characteristic state z according to the following formula. i (t+1) ,Right now ； Update the relational embedding state r according to the following formula: ij (t+1) ,Right now ， Among them, MLP u (·) and MLP r (·) represents the parameter update of the multilayer perceptron for the node state and the relationship embedding state, respectively.

8. The method for predicting the risk of gastric cancer lymph node metastasis based on spatially aware multi-instance learning according to claim 7, characterized in that, The output of the lymph node metastasis risk probability corresponding to this staining type includes: After T iterations of message passing, the graph features of the entire slice are integrated into a global feature vector Z through global pooling operations in the readout layer. global The expression is ； The global feature vector is input into a linear classifier, and the lymph node metastasis risk probability y corresponding to this staining type is calculated using an activation function. * The expression is ， Among them, W o With b o These are the weight matrix and bias parameters of the output linear classification layer, respectively.

9. The method for predicting the risk of gastric cancer lymph node metastasis based on spatially perceptive multi-instance learning according to claim 8, characterized in that, The step of performing probability-level fusion of the lymph node metastasis risk probabilities corresponding to all staining types to obtain the final lymph node metastasis risk assessment result for the patient under test includes: The AI ​​immune score (AIS) is calculated using the following formula: ， Where M is the number of staining types, P m (Y=1|X) m ) represents the positive confidence probability value of lymph node metastasis output by the multi-instance learning predictor of the side graph relationship corresponding to the m-th staining type; The cutoff value is determined based on the maximum point of the Youden index on the receiver operating characteristic curve. When the AIS is greater than the cutoff value, the patient is determined to be positive for lymph node metastasis; otherwise, the patient is determined to be negative for lymph node metastasis.

Images