System for realizing precise prognosis prediction of gastric cancer patient based on deep learning

Abstract

The invention relates to the technical field of pathological section image recognition, in particular to a system for achieving precise prognosis prediction of a gastric cancer patient based on deep learning, and aims at achieving precise prognosis prediction of the gastric cancer cell tumor patient on the pathological section level. The implementation package of the system comprises an information acquisition module, an image processing module, a wavelet parallel Mama model module, a result prediction module and an interpretability module, so that the efficiency of image feature extraction and the accuracy of model prediction are greatly improved. The core target of the invention is to solve the problems of insufficient accuracy, poorer result stability, limited model generalization ability, lack of interpretability and the like in the existing gastric cancer prognosis prediction method. Through strict verification and testing, the wavelet parallel Mama model of the research shows excellent performance in the aspect of predicting the total lifetime of gastric cancer patients in multiple centers, provides reliable support for clinical decision-making, and manifests the huge potential of the wavelet parallel Mama model in application in the field of precision medical treatment.

Description

Technical Field

[0001] The present invention relates to the technical field of pathological section image recognition, and in particular to a system for accurately predicting the prognosis of gastric cancer patients based on deep learning. Background Art

[0002] Gastric cancer (GC) remains a major public health challenge. Despite improvements in mortality due to better medical care and lifestyle changes, GC still poses a serious threat, and there is an urgent need to strengthen screening and prevention strategies to address this growing public health burden.

[0003] Gastric cancer exhibits remarkable histological diversity, as evidenced by the varying cell morphology and histological arrangement observed on hematoxylin and eosin (H&E) staining. Different subtypes of gastric cancer exhibit distinct architectural features, impacting their diagnosis and treatment. For example, tubular adenocarcinoma, the most common subtype, typically presents with irregularly dilated and branching tubules filled with intraluminal mucin, displaying a well-defined glandular architecture. In contrast, less cohesive carcinomas, including signet ring cell carcinoma, are characterized by cells that are either solitary or arranged in small aggregates without well-formed glands and often contain abundant mucin, which replaces the nucleus, resulting in the typical signet ring appearance. Diffuse gastric cancer exhibits a more invasive pattern, with tumor cells dispersed throughout the tissue, lacking organizational structure and often accompanied by necrosis and active mitosis. Furthermore, mucinous adenocarcinoma is defined by a pool of extracellular mucin that constitutes a significant portion of the tumor volume, further complicating its histological classification. This heterogeneity in cell morphology and histological arrangement among different subtypes underscores the complexity of gastric cancer and the need for precise histopathological assessment to guide effective treatment strategies.

[0004] The prognosis of gastric cancer varies significantly among its different subtypes, each of which has different histological features that can serve as potential prognostic biomarkers. For example, the Epstein-Barr virus subtype is associated with the best prognosis, while the genomically stable subtype is associated with the worst prognosis. Other subtypes, such as microsatellite instability and chromosomal instability, present with intermediate survival rates. Identifying specific morphological features observed in hematoxylin and eosin staining, such as cellular arrangement and nuclear atypia, is crucial for predicting patient prognosis. Given the complexity of these histological features, artificial intelligence (AI) has emerged as a promising solution to improve diagnostic accuracy and prognostic prediction. By analyzing large datasets of histopathological images, AI can potentially reveal subtle patterns that traditional methods may overlook, thereby improving patient stratification based on their expected survival and guiding personalized treatment approaches.

[0005] While researchers are increasingly applying artificial intelligence techniques, such as convolutional neural networks, multiple instance learning, and visual transformers, to pathology image diagnosis, these approaches face several significant challenges. A major limitation is their ability to effectively analyze the complex features of the immune microenvironment, where the interactions between different cell types and their spatial arrangement are crucial for understanding tumor biology. Furthermore, while some AI models excel at capturing long-range dependencies in images, they often require significant computational resources, which can hinder their practicality in clinical settings. The sparsity of annotated data further complicates the issue, as the availability of many high-quality annotated datasets is limited, making it difficult to train models that generalize well across a wide range of conditions. Furthermore, the interpretability and transparency of AI-driven decisions remain key concerns; clinicians need to understand how these models reach their conclusions in order to trust and integrate them into diagnostic workflows. Addressing these shortcomings is crucial to advancing the application of AI in pathology and ensuring its effectiveness in improving patient outcomes.

[0006] Combining multiple omics analyses, particularly by combining pathogenomics with immune infiltration analysis or spatial transcriptomics, provides a powerful approach for effectively analyzing the complex features of the immune microenvironment in cancer. Using techniques such as spatial transcriptomics, researchers can gain deep insights into the cellular composition and spatial organization of immune cells within tumors. This multidimensional analysis can identify specific immune cell types, their interactions, and signaling pathways that influence tumor progression and treatment response. For example, spatial transcriptomics retains the two-dimensional positional information of cells while providing gene expression data, which allows us to fully understand how different regions within the tumor microenvironment participate in the immune response. In addition, combining these methods can reveal key prognostic biomarkers by highlighting unique transcriptional signatures associated with various immune cell states and their spatial arrangements.

[0007] Furthermore, wavelet transforms and the parallel Mamba module are cutting-edge models in the field of artificial intelligence, particularly in image processing and analysis. These models are designed to be user-friendly, making them easy for researchers and practitioners to operate. They meet the necessary requirements for deployment on cloud platforms and scanners, facilitating seamless integration with existing workflows. Wavelet transforms enhance the ability to capture both low- and high-frequency components of images, while the Mamba module optimizes the processing of these features through advanced attention mechanisms. The combination not only improves image quality and training convergence, but also supports the translation of scientific research into practical applications. This combination enables more efficient analysis of complex datasets, paving the way for advancements in diverse fields such as medical imaging and diagnostics.

[0008] This paper describes a wavelet-based parallel Mamba model module based on full-field digital pathology slides, aiming to achieve accurate prognostic assessment and model interpretability for gastric cancer patients. Designed with the needs of modern medical image analysis in mind, this model combines residual connections, a parallel Mamba module, wavelet transforms, and a feature fusion module to achieve efficient and accurate analysis.

[0009] The wavelet-parallel Mamba model preprocesses pathological slides using wavelet transforms, decomposing images into subbands of varying scales. This model then extracts a wide range of features, from subtle cellular structures to overall tissue layout. This approach enables the model to simultaneously identify diffusion patterns associated with gastric cancer and detect micromorphological changes, building a rich dataset and enhancing the comprehensiveness and accuracy of the analysis. By introducing parallel processing Mamba layers and a state-space model, the model significantly improves computational efficiency, making it well-suited for clinical settings requiring rapid and reliable prognostic information. Furthermore, the model incorporates residual connections and a feature fusion module to effectively maintain data integrity and mitigate the effects of information degradation. The feature fusion module dynamically adjusts the weights of different features to suit the research objectives and data characteristics, thereby enhancing the model's flexibility and accuracy in prognostic prediction. This design not only reduces reliance on manual feature engineering but also effectively handles noisy and incomplete data, ensuring the model's robustness across a wide range of scenarios. The ability to automatically generate key features significantly improves the model's sensitivity and specificity in predicting gastric cancer outcomes, providing strong data support for the development of personalized treatment plans. In summary, the wavelet parallel Mamba model not only optimizes the pathological image analysis process through advanced technical means, but also provides more efficient and accurate tools for clinical applications, promoting the development of precision medicine. Summary of the Invention

[0010] To address the current problems of low accuracy, poor result stability, weak model generalization ability and poor interpretability in prognosis prediction for gastric cancer patients, this study proposed a new deep learning model architecture called the wavelet parallel Mamba model. This model combines wavelet transform, parallel Mamba layers, residual connections and feature fusion modules to achieve a very large receptive field without over-parameterization, effectively capturing local and global information of the image.

[0011] First, a wavelet parallel Mamba model based on full-field digital pathology slides was designed for accurate prognosis prediction of gastric cancer patients. The system mainly consists of five modules: information acquisition module, image processing module, wavelet parallel Mamba model module, result prediction module and interpretability module:

[0012] The information acquisition module is responsible for performing high-resolution scanning of pathological sections of gastric cancer patients and generating full-field digital pathological section images including the tumor area;

[0013] The image processing module performs preprocessing on the full-field image, including image segmentation, slicing and staining correction, to generate standardized image blocks for subsequent analysis;

[0014] The wavelet parallel Mamba model module adopts wavelet transform, parallel Mamba layer, residual connection and feature fusion technology. Specifically, it includes: (1) Wavelet transform: including discrete wavelet transform, convolution processing, inverse discrete wavelet transform and the generation of final feature map, using multi-level wavelet transform to decompose feature map and capture macroscopic and microscopic pathological features in the image. (2) Parallel Mamba layer: including input normalization, data reshaping, Mamba state space model, splicing processed data, re-normalization, linear projection and data restoration to original shape, realizing efficient processing of global features through state space model, and using lightweight parallel visual processing layer to improve computational efficiency. (3) Residual connection and feature fusion module: including feature map fusion, global average pooling, attention map calculation, feature map weighting, channel dimensionality reduction, attention modulation and final feature fusion, combining residual connection and feature fusion to ensure the stability of information flow and realize adaptive feature fusion.

[0015] The outcome prediction module uses the negative partial log-likelihood loss to predict patient survival risk based on the fused feature set and optimizes the training results. The system outputs a C-index score for the predicted score through five-fold cross-validation and external test cohort validation.

[0016] The interpretability module outputs an attention score and risk score through the model. The system achieves model interpretability by performing attention heatmaps, t-SNE dimensionality reduction clustering, and receptive field calculations. Furthermore, it conducts multi-omics analyses, including spatial transcriptomics, high-throughput transcriptomics, and whole-exome analyses, to explore the biological processes associated with risk scores. Ultimately, the output risk score and clinical indicators of gastric cancer patients, including stage, grade, age, and gender, are combined to construct a list for predicting patient survival, which is then deployed on slide scanners and online platforms.

[0017] This application discloses a system that uses a wavelet-parallel Mamba model to accurately predict the prognosis of gastric cancer patients at the level of routine pathological sections. By combining clinical indicators and risk scores, a nomogram is created that is easy for clinicians to use and understand. This system has improved consistency, sensitivity, specificity, and interpretability, while significantly reducing the financial burden on patients, facilitating precise diagnosis and treatment of gastric cancer patients.

[0018] The system disclosed in the present invention can accurately predict the prognosis of gastric cancer patients at the level of conventional pathological sections, and can be deployed on cloud platforms and scanners, reducing the economic burden on patients and helping to promote accurate diagnosis and treatment of gastric cancer patients.

[0019] This paper adopts three visualization methods to achieve model interpretability, including attention heat map drawing, t-SNE dimensionality reduction clustering and receptive field calculation, which improves the transparency of model decision-making.

[0020] The present invention combines multi-omics data such as spatial transcriptome, high-throughput transcriptome, and whole exome, overcoming the limitations of artificial intelligence analysis of the complexity of the immune microenvironment.

[0021] The present invention introduces an innovative negative partial log-likelihood loss (NLLSurvLoss) to accurately predict patient survival time and perform risk assessment.

[0022] The wavelet parallel Mamba model involved in the present invention implements wavelet transform through the WTConv layer, provides effective multi-frequency response, and expands smoothly as the receiving domain expands.

[0023] The feature fusion module in the wavelet parallel Mamba model involved in the present invention solves the potential degradation effect caused by residual connections and ensures efficient transmission of information flow.

[0024] The wavelet parallel Mamba model involved in the present invention adopts a PVMLayer for parallel processing of visual Mamba features, achieving high-performance feature extraction with low computational complexity while maintaining the total number of channels. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] Figure 1 The overall flow chart of prognosis prediction for gastric cancer patients based on wavelet parallel Mamba model;

[0027] Figure 2 Algorithm 1: Algorithm flow for preprocessing digital pathology slides and constructing label information;

[0028] Figure 3 Algorithm 2 Wavelet parallel Mamba model development, training and verification;

[0029] Figure 4 Wavelet parallel Mamba model mechanism diagram;

[0030] Figure 5 Attention heatmap of the wavelet parallel Mamba model;

[0031] Figure 6 t-SNE dimensionality reduction clustering diagram of the wavelet parallel Mamba model;

[0032] Figure 7 Receptive field calculation diagram of the wavelet parallel Mamba model;

[0033] Figure 8 Nomogram of model risk score combined with clinical indicators. DETAILED DESCRIPTION

[0034] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention. The technical solutions provided by each embodiment of the present invention are described in detail below in conjunction with the drawings.

[0035] The following is a system for accurately predicting the prognosis of gastric cancer patients based on deep learning involved in the present invention ( Figure 1 ), including the following sub-modules:

[0036] S1, information collection module;

[0037] S2, image processing module;

[0038] S3, wavelet parallel Mamba model module;

[0039] S4, result prediction module;

[0040] S5. Explainability module.

[0041] The information collection module includes the following inclusion criteria for gastric cancer patients: the patient is older than 18 years of age at the time of diagnosis; the diagnosis of gastric cancer is confirmed by a qualified pathologist according to the latest World Health Organization classification standards for digestive system tumors; the patient does not suffer from severe cardiovascular disease or psychiatric illness; the patient has complete follow-up data, and their pathological tissue sections and paraffin blocks are well preserved; the patient's survival status includes death, survival, or censoring, among which the starting point for calculating overall survival is the date of pathological diagnosis, and the end point is the date of death or the last follow-up date of the patient.

[0042] The information collection module includes the following information: basic clinical information of gastric cancer patients, including age, gender, pathological grade, lesion location, medical history, treatment history, family genetic history, and biomarker status (including Ki-67, PD-L1, HER-2, CerB-2, CDX2, P53, etc.); follow-up data of patients, including diagnosis date, disease progression and time, survival status (death or survival), date of death or last follow-up date; full-field digital pathology sections of pathological tissue after fixation, embedding and staining, image data obtained using a high-magnification digital scanner, and image blocks extracted from them for subsequent analysis.

[0043] The image processing module preprocesses digitized full-field slice images of gastric cancer patients, mainly including: segmentation: removing useless background in the image; slicing: cutting the image into multiple non-overlapping 256×256 pixel image blocks; feature extraction: using pre-trained large language models in the pathology field (UNI, CONCH, RESNET) to extract the feature vector of each image block.

[0044] Patients were followed up by telephone. Survival status was categorized as death, survival, or censored. Overall survival was calculated starting from the date of pathological diagnosis and ending at the date of death or the last follow-up. Follow-up information for all gastric cancer patients in the cohort was obtained through telephone follow-up. Gastric cancer patient pathology slides were digitally scanned at 40x magnification using a digital pathology slide scanner. Image blocks were extracted from the digital pathology slides to capture the complex structural information in the digital pathology slides. For details (see the corresponding algorithm flow), Figure 2 Design and implement a wavelet parallel Mamba model that fuses local and global information, captures local information through a global to local spatial aggregation module, and processes the Mamba layer in parallel to capture global information for feature fusion (see the Wavelet Parallel Mamba Model Development and Training Algorithm Process for details). Figure 3 ).

[0045] The model designed in this paper is a wavelet parallel Mamba model that focuses on the analysis of digital pathology slides, aiming to solve the problem of prognosis assessment for gastric cancer patients. The model design integrates modern artificial intelligence technologies such as wavelet transform, parallel processing Mamba layer and feature fusion module, and has the following core components: Figure 4 ):

[0046] Wavelet transform: To overcome the parameter overload problem of increasing the convolution kernel size to simulate the global receptive field, this paper uses a wavelet transform method. This method can achieve extremely large receptive fields without over-parameterization. For a k×k receptive field, the number of trainable parameters in the algorithm grows only logarithmically with k. The proposed layer, called WTConv, can replace existing architectures, produce effective multi-frequency response, and scale gracefully with the expansion of the receptive field;

[0047] Parallel Mamba Layer: This paper introduces a lightweight visual Mamba UNet, in which the parallel Mamba layer can process visual Mamba features in parallel, achieving faster computation speed while keeping the total number of channels unchanged;

[0048] Residual connections: To enhance the stability and performance of deep network training, residual connections are added to the model. This design allows signals to be passed directly from the previous layer to the next layer while preserving high-level abstract features;

[0049] Feature fusion: Considering that residual connections may cause degradation effects in deep networks, this paper introduces a feature fusion module that can adaptively combine local and global features to solve the problem of unnatural visual perception.

[0050] The parallel processing Mamba layer, residual connection, wavelet transform and feature fusion module of the image training module are specifically implemented as follows:

[0051] 1. Wavelet transform: including discrete wavelet transform, convolution processing, inverse discrete wavelet transform, and generation of the final feature map:

[0052] (1) Discrete wavelet transform:

[0053] (1)

[0054] Given a wavelet filter pair ( , ) and the input data tensor ,in is the batch size, is the number of channels, and are the height and width of the image. Here, Indicates the The low-frequency subband of the layer, is the index of the filter that is convolved with the input data by inversion and translation. This process is applied to all subbands, producing , , .

[0055] (2) Convolution processing:

[0056] (2)

[0057] Apply convolution operation on each subband after wavelet transformation. , , , Perform the above operations. are the weights of the convolutional layer, Represents a two-dimensional convolution operation.

[0058] (3) Inverse discrete wavelet transform:

[0059] (3)

[0060] (4)

[0061] Given a wavelet filter pair ( , ), combine these convolved sub-bands and apply inverse wavelet transform to reconstruct the data. And so on, reconstruct all sub-bands and get , , .

[0062] (4) Generation of the final feature map:

[0063] (4)

[0064] Finally, the inverse transformed subbands are combined back into the original image space and added to the output of the original convolutional layer to generate the final feature map. is the output of the model, is the weight of the base convolutional layer, is the number of wavelet transform layers.

[0065] 2. Parallel Mamba layer: including input normalization, data reshaping, Mamba state space model, splicing processed data, re-normalization, linear projection, and data restoration to its original shape:

[0066] (1) Input normalization:

[0067] (5)

[0068] Given an input data tensor ,in is the batch size, is the number of channels, and are the height and width of the image. and Respectively The mean and standard deviation of .

[0069] (2) Data reshaping:

[0070] (6)

[0071] (7)

[0072] Continue to normalize Reshape into ,in is the total number of pixels in the image and is represented by Processing converts it into To match the format required by the model. (3) Mamba state space model:

[0073] (8)

[0074] (9)

[0075] After the remodeling Divided into four parts , each part is passed The model is processed. is the state-space model operation, is the scaling parameter of the skip connection.

[0076] (4) Data after splicing:

[0077] (10)

[0078] Will The four parts of the model are spliced back together after processing.

[0079] (5) Normalize again:

[0080] (11)

[0081] After splicing Normalize again. and Respectively The mean and standard deviation of .

[0082] (6) Linear projection:

[0083] (12)

[0084] The normalized data is projected through a linear layer. and are the weights and biases of the linear layer respectively.

[0085] (7) Data restored to its original shape:

[0086] (13)

[0087] Finally, the data is restored to the original image shape.

[0088] 3. Residual Connection

[0089] (14)

[0090] In actual situations, It represents the result of a series of operations such as convolution and nonlinear activation functions.

[0091] 4. Feature fusion module: including feature map fusion, global average pooling, attention map calculation, feature map weighting, channel dimensionality reduction, attention modulation, and final feature fusion:

[0092] (1) Feature map fusion:

[0093] (15)

[0094] Given two feature maps and ,in is the batch size, , is the number of channels of the feature map, , , The depth, height and width of the feature map are concatenated along the channel direction to generate a new feature map. .

[0095] (2) Global average pooling:

[0096] (16)

[0097] Next, adaptive global average pooling is applied to the concatenated feature maps. , to extract global context information. Here is a single vector per batch.

[0098] (3) Attention map calculation:

[0099] (17)

[0100] Using a convolutional layer and Activation function, calculate an attention map , which describes the importance of each channel. .

[0101] (4) Feature map weighting:

[0102] (18)

[0103] Attention map and concatenated feature maps Multiply and weight the feature map. Here , Represents element-wise multiplication.

[0104] (5) Channel dimensionality reduction:

[0105] (19)

[0106] The number of channels of the feature map is reduced by another convolutional layer. Here .

[0107] (6) Attention modulation:

[0108] (20)

[0109] (twenty one)

[0110] In addition, through two different convolutional layers and The activation function calculates two additional attention maps, which weight the original feature map respectively. Here , .

[0111] (7) Final feature fusion:

[0112] (twenty two)

[0113] Finally, the feature map after channel dimension reduction is multiplied with the two weighted feature maps to obtain the final feature fusion result. is the fused feature map, which is the output of the module.

[0114] 5. The hyperparameter settings for model training are as follows: During model training, this experiment selected the Adam optimizer, set the learning rate to 0.002, the batch size to 1, and the number of training rounds to 50 epochs.

[0115] 6. Model result output: After model training is completed, the fused feature information of each image block can be output. This feature information contains rich image and tissue information. Then, through classifier processing, the predicted risk value is obtained. Hazards are calculated and the sigmoid function is used to map the risk value to the (0, 1) interval. Finally, the cumulative product is used to calculate the survival function. In survival analysis, we will use the negative partial log-likelihood as the loss function based on the Cox proportional hazard model. It measures the difference between the risk value predicted by the model and the actual observed survival time. The specific loss function is as follows:

[0116] (1) Cumulative product of survival function:

[0117] (twenty three)

[0118] here Indicates in The probability of survival at the end of the time interval.

[0119] (2) Filling survival function:

[0120] (twenty four)

[0121] here is the filled survival function, considering that all patients are alive during the period of time.

[0122] (3) Loss of untruncated samples:

[0123] (25)

[0124] here is the sum of the losses of all untruncated samples.

[0125] (4) Loss of truncated samples:

[0126] (26)

[0127] here is the sum of the losses of all truncated samples.

[0128] After the above training, the wavelet parallel Mamba model involved in the present invention achieved accurate prediction of the overall survival of gastric cancer patients in the TCGA-STAD internal validation and ZN-STAD external validation cohorts (Table 1).

[0129] Table 1 The wavelet parallel Mamba model accurately predicted the overall survival of gastric cancer patients in the TCGA-STAD internal validation cohort and the ZN-STAD external validation cohort

[0130] TCGA-STAD ZN-STAD C-Index (%) 72.48±0.068 70.63±0.077

[0131] 7. Model Interpretability: To achieve model interpretability, the wavelet parallel Mamba model involved in this invention outputs attention scores after training. Using these output attention scores, we will improve model interpretability through three visualization methods: attention heatmaps, t-SNE dimensionality reduction clustering, and receptive field calculation.

[0132] (1) Attention heatmap: The attention heatmap is used to visualize the contribution of different regions of the input image to the model output; for the preprocessed image block, its attention score is expressed as:

[0133] (27)

[0134] in It's location The attention score, is the calculated feature response. Finally, the attention heatmap is obtained by combining the attention score with the original image:

[0135] (28)

[0136] in, is the generated heat map, is the original image at position Pixel values (see the attention heat map of the wavelet parallel Mamba model Figure 5 ).

[0137] (2) t-SNE dimensionality reduction clustering: t-SNE (t-distributed stochastic neighbor embedding) is a technique for dimensionality reduction of high-dimensional data. Its goal is to embed data points in a high-dimensional space into a low-dimensional space while maintaining the relative distance between data points. First, prepare the feature matrix extracted from the pathological image block:

[0138] (29)

[0139] in, is the number of image blocks, is the original feature dimension. Then perform t-SNE dimensionality reduction:

[0140] (30)

[0141] in, Indicates that at a given point In the case of The probability of being selected as a neighbor; is and dot The standard deviation of the associated Gaussian distribution. In low-dimensional space, t-SNE uses the following formula to define similarity:

[0142] (31)

[0143] here, Indicates the midpoint in low-dimensional space and point For each image block Perform K-Means clustering and evaluation:

[0144] (32)

[0145] (33)

[0146] in, express The average distance to other points in the same cluster, express The average distance to the nearest other cluster points, the global evaluation value is all image blocks Maximize this value to select the optimal number of clusters . Then visualize:

[0147] (34)

[0148] in, is the generated cluster graph, Each image patch is assigned a cluster label, is the original image at position Pixel values (see t-SNE dimensionality reduction clustering diagram of wavelet parallel Mamba model Figure 6 ).

[0149] (3) Receptive field calculation: The receptive field is the size of the input image area corresponding to a pixel on each feature map in the neural network. First, the model is forward propagated to obtain the output of the model. :

[0150] (35)

[0151] in is the multi-layer transformation of the model, High-dimensional feature tensor extracted for pathological image blocks, is the batch size, is the number of channels, is the image height, is the image width. Then, after the center point activation calculation, the features of the center point are obtained. :

[0152] (36)

[0153] in is the activation function, Represents the integer division calculation process. Then through the back propagation gradient calculation:

[0154] (37)

[0155] in is the feature dimension, is the gradient of the center point to the input. Then the gradient processing and contribution aggregation are performed:

[0156] (38)

[0157] (39)

[0158] in is the activation function (see the receptive field calculation diagram of the wavelet parallel Mamba model Figure 7 ).

[0159] 8. Exploration of prognosis-related biological processes: To explore the prognosis-related biological processes predicted by the model, the wavelet parallel Mamba model involved in the present invention will output attention scores and risk scores after training. Using the output risk scores and interpretable results, multi-omics analysis is performed in conjunction with spatial transcriptome, high-throughput transcriptome, and whole exome:

[0160] (1) Combination of heat map and spatial transcriptome: For the poor prognosis-related regions and good prognosis-related regions identified by the heat map, spatial transcriptome data are used to obtain the characteristic genes of the corresponding regions, and then the biological processes that are beneficial and harmful to the prognosis of gastric cancer patients are identified.

[0161] (2) Combination of risk scores and high-throughput transcriptomics: Using Cibersort, EPIC, xCELL, and ESTIMATE immune infiltration algorithms to identify immune cell types associated with risk scores;

[0162] Identify gene modules co-expressed with the risk score for enrichment analysis and identify biological processes related to the survival prognosis of gastric cancer patients. Through enrichment analysis of co-expressed gene modules, the significance can be calculated using hypergeometric distribution:

[0163] (40)

[0164] in, is the total number of genes in the background gene set, is the total number of genes in the target gene set, is the number of significant genes in the target gene set, is the number of genes that appear in both sets.

[0165] (3) Combination of risk grouping and whole exome sequencing: Gastric cancer patients are divided into high-risk and low-risk groups based on the risk score, and the differences in mutation landscape between the two groups are identified. By comparing the differences in mutation frequencies between the high-risk and low-risk groups, statistical significance analysis can be performed using the chi-square test or Fisher's exact test:

[0166] (41)

[0167] in, is the observed frequency, is the expected frequency.

[0168] 9. Model Integration with Clinical Indicators and Deployment: To achieve model deployment and clinical translation, the wavelet parallel Mamba model involved in this invention will output an attention score after training. Using the output risk score, we will combine it with the clinical indicators of gastric cancer patients (including stage, grade, age, and gender) to construct a nomogram for predicting patient survival. This nomogram will be deployed on slide scanners and online platforms for use by clinicians and patients (see the constructed nomogram for details). Figure 8 ).

[0169] The wavelet parallel Mamba model constructed in the present invention provides an important tool for predicting overall survival in gastric cancer patients. While preferred embodiments of the present invention have been described, those skilled in the art will readily make additional changes and modifications to these embodiments once they understand the underlying inventive concepts. Therefore, the appended claims are intended to be interpreted as covering the preferred embodiments as well as all variations and modifications that fall within the scope of the invention.

[0170] Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if such modifications and variations fall within the scope of the claims and their equivalents, the present invention is intended to include such modifications and variations, and the above-described embodiments of the present invention do not constitute a limitation on the scope of protection of the present invention.

Claims

1. A system for accurately predicting the prognosis of gastric cancer patients based on deep learning, characterized in that: The system implementation includes the following modular steps: S1, information collection module; S2, image processing module; S3, the wavelet parallel Mamba model module, fuses local and global information and optimizes features through a feature fusion module. This model uses a parallel structure to simultaneously capture microscopic and macroscopic pathological features. The specific model structure includes the following submodules: wavelet transform submodule, parallel Mamba submodule, residual connection submodule, and feature fusion submodule. S4, result prediction module; S5. Explainability module.

2. The system for accurately predicting the prognosis of gastric cancer patients based on deep learning according to claim 1, characterized in that: The module S1, namely the information collection module, includes the following contents: setting the inclusion criteria of gastric cancer patients, collecting the basic clinical information of gastric cancer patients, and digitizing the pathological tissue sections.

3. The system for accurately predicting the prognosis of gastric cancer patients based on deep learning according to claim 1, characterized in that: The module S2, ie the image processing module, includes: performing segmentation, slicing and staining correction on the digitized image to generate standardized image blocks for subsequent analysis.

4. The system for accurately predicting the prognosis of gastric cancer patients based on deep learning according to claim 1, characterized in that: The module S3, i.e. the wavelet parallel Mamba model module, has a wavelet transform submodule comprising the following four steps: Step 1: Discrete wavelet transform: （1） Given a wavelet filter pair , and the input data tensor ,in is the batch size, is the number of channels, and are the height and width of the image; here, Indicates the The low-frequency subband of the layer, is the index of the filter, which is convolved with the input data by inversion and translation. This process is applied to all subbands, producing , , ; Step 2: Convolution processing: （2） Apply convolution operation on each subband after wavelet transformation, for each subband , , , Perform the above operations, where are the weights of the convolutional layer, Represents a two-dimensional convolution operation; Step 3: Inverse discrete wavelet transform: （3） （4） Given a wavelet filter pair , , combine these convolved sub-bands and apply inverse wavelet transform to reconstruct the data, and so on, reconstruct all sub-bands to obtain , , ; Step 4: Generation of the final feature map: （5） Finally, the inverse transformed subbands are combined back into the original image space and added to the output of the original convolutional layer to generate the final feature map, where is the output of the model, is the weight of the base convolutional layer, is the number of wavelet transform layers.

5. The system for accurately predicting the prognosis of gastric cancer patients based on deep learning according to claim 1, characterized in that: The module S3, i.e. the wavelet parallel Mamba model module, includes the following steps: Step 1: Input normalization: （6） Given an input data tensor ,in is the batch size, is the number of channels, and are the height and width of the image, and Respectively The mean and standard deviation of Step 2: Data reshaping: （7） （8） Continue to normalize Reshape into ,in is the total number of pixels in the image and is represented by Processing converts it into to match the format required by the model; Step 3, Mamba state space model: （9） （10） After the remodeling Divided into four parts , each part is passed The model is processed, is the state-space model operation, is the scaling parameter of the skip connection; Step 4: Splice the processed data: （11） Will The four parts of the model are stitched back together after processing; Step 5: Normalize again: （12） After splicing Normalize again, and Respectively The mean and standard deviation of Step 6: Linear projection （13） The normalized data is projected through a linear layer, where and are the weights and biases of the linear layer respectively; Step 7: Restore the data to its original shape: （14） Finally, the data is restored to the original image shape, where is the batch size, is the number of channels, is the image height, is the image width.

6. The system for accurately predicting the prognosis of gastric cancer patients based on deep learning according to claim 1, characterized in that: The module S3 is a wavelet parallel Mamba model module, and the residual connection submodule of the module is expressed as: （15） In actual situations, It represents the result of a series of operations such as convolution and nonlinear activation functions.

7. The system for accurately predicting the prognosis of gastric cancer patients based on deep learning according to claim 1, characterized in that: The module S3, namely the wavelet parallel Mamba model module, has a feature fusion submodule that operates in the following seven steps: Step 1: Feature map fusion: （16） Given two feature maps and ,in is the batch size, , is the number of channels of the feature map, , , is the depth, height and width of the feature map. These two feature maps are spliced together along the channel direction to generate a new feature map ; Step 2: Global average pooling: （17） Next, adaptive global average pooling is applied to the concatenated feature maps. , to extract global context information, here is a single vector per batch; Step 3. Attention map calculation: （18） Using a convolutional layer and Activation function, calculate an attention map , which describes the importance of each channel; ; Step 4: Feature map weighting: （19） Attention map and concatenated feature maps Multiply and weight the feature map, here , represents element-wise multiplication; Step 5: Channel dimensionality reduction: （20） The number of channels of the feature map is reduced by another convolutional layer; here ; Step 6: Attention modulation: （21） （22） In addition, through two different convolutional layers and The activation function calculates two additional attention maps, which weight the original feature map respectively. Here , ; Step 7: Final feature fusion: （23） Finally, the feature map after channel dimensionality reduction is multiplied by the two weighted feature maps to obtain the final feature fusion result. Here is the fused feature map, which is the output of the module.

8. According to the system for accurately predicting the prognosis of gastric cancer patients based on deep learning in claim 1, the module S4, i.e., the outcome prediction module, is characterized in that: After model training is completed, it can output the fused feature information of each image block and use the classifier to process these features to obtain the predicted risk value. The risk value is mapped between 0 and 1 through the sigmoid function, and the cumulative product method is used to calculate the survival function. A new negative partial log-likelihood loss is introduced to achieve accurate prediction of patient survival time and risk assessment. The loss function calculation method includes the following steps: Cumulative product of survival functions: （24） here Indicates in The probability of survival at the end of the time interval is Indicates the time interval The conditional probability of death of patients in the Fill in the survival function: （25） here is the filled survival function, considering that all patients are alive during the time period; Loss for untruncated samples: （26） here is the sum of the losses of all untruncated samples; Loss for truncated samples: （27） here is the sum of the losses of all truncated samples.

9. The system for accurately predicting the prognosis of gastric cancer patients based on deep learning according to claim 1, characterized in that: Module S5, the interpretability module, improves model interpretability through three visualization methods: attention heatmap, t-SNE dimensionality reduction clustering, and receptive field: Attention heatmap: The attention heatmap is used to visualize the contribution of different regions of the input image to the model output; for the preprocessed image patch, its attention score is expressed as: （28） in It's location The attention score, is the calculated feature response; finally, the attention heatmap is obtained by combining the attention score with the original image: （29） in, is the generated heat map, is the original image at position Pixel value of t-SNE dimensionality reduction clustering: First, prepare the feature matrix extracted from the pathological image block: （30） in, is the number of image blocks, is the original feature dimension; then t-SNE dimension reduction is performed: （31） in, Indicates that at a given point In the case of The probability of being selected as a neighbor; is and dot The standard deviation of the associated Gaussian distribution; in low-dimensional space, t-SNE uses the following formula to define similarity: （32） here, Indicates the midpoint in low-dimensional space and point Similarity; for each image block Perform K-means clustering and evaluation: （33） （34） in, express The average distance to other points in the same cluster, express The average distance to the nearest other cluster points, the global evaluation value is all image blocks Maximize this value to select the optimal number of clusters ; Then visualize: （35） in, is the generated cluster graph, Each image patch is assigned a cluster label, is the original image at position Pixel value of Receptive field calculation: The receptive field is the size of the input image area corresponding to a pixel on each feature map in the neural network; first, the model is forward propagated to obtain the output of the model : （36） in is the multi-layer transformation of the model, High-dimensional feature tensor extracted for pathological image blocks, is the batch size, is the number of channels, is the image height, is the image width; then the center point activation calculation is performed to obtain the center point feature : （37） in is the activation function, Represents the integer division calculation process, and then back-propagation gradient calculation: （38） in is the feature dimension, is the gradient of the center point to the input; gradient processing and contribution aggregation are then performed: （39） （40） in is the activation function.

10. The system for accurately predicting the prognosis of gastric cancer patients based on deep learning according to claim 1, characterized in that: Module S5, the interpretability module, also includes multi-omics analysis using the risk scores output by the model in conjunction with spatial transcriptome, high-throughput transcriptome, and whole exome to further achieve model interpretability from a biological perspective. Specifically, it includes the following four aspects: Combining risk scores and spatial transcriptomes: A higher risk score indicates a worse prognosis for the patient. The corresponding risk score is visualized as a heat map. For regions associated with poor and good prognosis identified by the heat map, spatial transcriptome data is used to obtain characteristic expression genes in the corresponding regions. This allows identification of biological processes that are beneficial or detrimental to the prognosis of gastric cancer patients, thereby achieving model interpretability at the gene expression level. Combining risk scores with high-throughput transcriptomics: Using Cibersort, EPIC, xCELL, and ESTIMATE, four gene transcriptome data algorithms are used to infer immune cell infiltration, identifying immune cell types associated with risk scores and thereby achieving model interpretability at the level of immune cell infiltration. Combination of risk grouping and whole exome sequencing: Gastric cancer patients are divided into high-risk and low-risk groups based on the risk score, and the differences in mutation landscape between the two groups are identified, thereby achieving model interpretability at the gene mutation level. The differences in gene mutations between different risk groups are statistically significant using at least one of the chi-square test and Fisher's exact test.

11. The system for accurately predicting the prognosis of gastric cancer patients based on deep learning according to claim 1, characterized in that: The risk score output by the model and the clinical indicators of gastric cancer patients were used to construct a nomogram for predicting patient survival, and it was deployed on slide scanners and online platforms for use by clinicians and patients.

Images