A method for realizing precise prognosis prediction of glioblastoma patients

Abstract

The present application relates to pathological section image recognition technical field, specifically to a kind of method for realizing glioblastoma patient precision prognosis prediction, propose a kind of multi-resolution attention multi-bag reasoning framework, aims at realizing the precise evaluation of prognosis of glioblastoma patient on pathological section level.Combined with resolution self-adaptive adjustment module, channel-space attention fusion module, spatial conical pooling module and exponential weighted smoothing module, the generalization ability and prediction accuracy of framework are greatly improved and solve the problems, such as low accuracy, weak generalization ability and poor transparency, existing in the prognosis evaluation method of glioblastoma.After strict verification, the multi-resolution attention multi-bag reasoning framework of the present application shows excellent performance in the evaluation of overall survival of patients in multiple centers, provides strong data support for the development of individualized diagnosis and treatment plan, and shows its great potential to promote the development of precision medicine.

Description

Technical Field

[0001] This invention relates to the field of pathological slide image recognition technology, and in particular to a method for accurate prognostic prediction of glioblastoma patients. Background Technology

[0002] Glioblastoma multiforme (GBM) is a significant public health challenge. Despite some limited improvements in survival resulting from advances in neuro-oncology and comprehensive treatment strategies, GBM remains the most common and aggressive primary malignant brain tumor in adults, posing an extremely serious threat to life. Its incidence is stable, but its prognosis is very poor, highlighting the urgent need for enhanced early diagnosis and prevention strategies to address this heavy public health burden.

[0003] GBM is renowned for its significant histological heterogeneity, manifested in highly diverse cellular morphologies and structural patterns in hematoxylin and eosin (H&E) stained sections. Different histological variants exhibit unique microscopic features, profoundly impacting their diagnosis and clinical management. For example, giant cell GBM is characterized by numerous large, bizarrely shaped multinucleated tumor giant cells with active mitotic figures, often accompanied by abundant reticular stroma. Gliosarcoma, on the other hand, exhibits a biphasic differentiation pattern, containing both typical malignant glioma components and clearly defined sarcomatoid mesenchymal tissue regions, forming a distinctive "collision tumor" appearance. Epithelioid GBM tumor cells possess a relatively uniform epithelioid morphology, with abundant, eosinophilic cytoplasm, prominent nucleoli, and arranged in sheet-like, nest-like, or glandular structures, sometimes leading to misdiagnosis as metastatic cancer. Furthermore, commonly observed characteristic structures of GBM include: significant tumor cell pleomorphism, high-density mitotic figures, geographic necrosis, and significant microvascular proliferation. The extreme heterogeneity in cell morphology and structural arrangement among different variants and within the same tumor highlights the inherent complexity of GBM and underscores the necessity of meticulous and precise histopathological evaluation for developing effective treatment strategies.

[0004] GBM generally has a very poor prognosis, but there are differences among its various molecular subtypes and under the influence of key molecular markers. These molecular features are partially associated with its histological appearance and can serve as important prognostic and treatment predictive biomarkers. For example, IDH wild-type GBM has the worst prognosis, while the relatively rare IDH mutant GBM has a relatively better prognosis. Within IDH wild-type GBM, subtypes defined based on extensive molecular profiling also show prognostic differences: classic, mesenchymal, and anterior neural types. Identifying specific morphological features observed in H&E staining, such as the extent and pattern of necrosis, the degree of microvascular proliferation, cell density, and the severity of nuclear atypia / pleomorphism, combined with molecular subtyping, is crucial for more accurately predicting patient prognosis and guiding treatment selection. Given the high complexity, variability, and subjective nature of GBM histological morphology, the application of artificial intelligence (AI) in digital pathology image analysis presents a promising solution for improving diagnostic consistency, objectively quantifying histological features, integrating molecular information, and ultimately enhancing the accuracy of prognostic prediction. By analyzing massive amounts of whole-slide images (WSI) using deep learning algorithms, AI can potentially reveal subtle morphological patterns, spatial distribution characteristics, and their complex relationships with molecular alterations, treatment responses, and survival outcomes that are difficult for the human eye to detect. This could optimize patient risk stratification and drive truly personalized treatment decisions.

[0005] Multiple bag inference (MBI) is a classic framework for applying artificial intelligence to pathomics. This framework is designed to be user-friendly, allowing researchers and practitioners to easily operate it. It meets the necessary requirements for deployment on cloud platforms and scanners, facilitating seamless integration with existing workflows. In addition to MBI, spatial cone pooling (ASPP) and exponentially weighted smoothing (EMA) are cutting-edge components in the field of artificial intelligence. ASPP can capture contextual information from receptive fields of different resolutions, enhancing the framework's ability to extract multi-resolution features. EMA aggregates features through exponentially weighted averaging, helping to smooth feature representations and potentially improving the framework's generalization ability. Integrating these two components into the MBI framework will make the framework more effective for analyzing complex datasets and is more likely to be translated into practical applications.

[0006] This invention relates to a multi-resolution attention multi-bag inference framework (MRA-MBIF) based on full-view digital pathology slides, designed to achieve accurate prognostic assessment and framework transparency for GBM patients. The framework is designed to meet the needs of modern medical image analysis, combining a resolution adaptive adjustment module, a channel-spatial attention fusion module, ASPP, and EMA to achieve efficient and accurate analysis.

[0007] The multi-resolution attention multi-baggage inference framework deeply optimizes pathological image analysis through its unique multi-module integration. First, it develops a novel resolution adaptive adjustment module capable of intelligently detecting and adaptively adjusting feature representations at different resolutions. Then, utilizing ASPP, the framework effectively extracts rich contextual information from the multi-resolution receptive field, and combines this with a channel-spatial attention fusion module to dynamically focus on the most important channels and spatial regions in the feature map. This multi-level feature extraction and weighting strategy enables the framework to simultaneously identify key patterns and microscopic morphological changes in pathological images, from minute cellular structures to overall tissue layout, constructing a more comprehensive and accurate feature representation. To further enhance feature stability and expressive power, the multi-resolution attention multi-baggage inference framework incorporates EMA for feature aggregation and, through its optimized feature extraction structure, introduces residual connections and SE-like attention mechanisms, effectively maintaining data integrity and mitigating the degradation effects of information in deep networks. This design not only reduces reliance on manual feature engineering but also effectively handles noise and incomplete data, ensuring the framework's robustness under various complex conditions. Ultimately, the multi-resolution attention multi-bag inference framework aggregates features from individual image segments into bag-level representations via the MBI mechanism and utilizes an enhanced decision-maker for prediction. This decision-maker incorporates temperature scaling and label smoothing techniques during the training phase, further enhancing the framework's generalization ability and prediction accuracy. This ability to automatically generate key features significantly improves the framework's detection sensitivity and specificity in pathological image analysis, providing strong data support for the development of personalized diagnosis and treatment plans and promoting the advancement of precision medicine. Summary of the Invention

[0008] To address the issues of low accuracy, weak generalization ability, and poor transparency in current prognostic predictions for GBM patients, this study proposes a novel deep learning framework architecture called the multi-resolution attention multi-bag inference framework. This framework combines a resolution adaptive adjustment module, a channel-spatial attention fusion module, ASPP, and EMA, which can effectively capture key patterns and micromorphological changes in images, from fine cellular structures to overall tissue layout.

[0009] First, a multi-resolution attention-based multi-baggage inference framework based on full-view digital pathological slides was designed for accurate prognostic assessment of GBM patients. This framework mainly consists of five modules: data acquisition and standardization, image preprocessing and standardization, the multi-resolution attention-based multi-baggage inference framework module, prediction output, and interpretive analysis.

[0010] The data acquisition and standardization module is responsible for the inclusion criteria of glioblastoma patients, collecting patients' basic clinical data, and performing high-resolution scanning of pathological slides of GBM patients to generate clear full-view digital pathological slide images.

[0011] The image preprocessing and standardization module preprocesses the full-view image, including image segmentation, cutting, and coloring standardization, to generate uniform image fragments for subsequent framework training.

[0012] The multi-resolution attention multi-bag inference framework module employs a resolution adaptive adjustment module, a channel-spatial attention fusion module, an ASPP module, and an EMA module. Specifically, it includes: (1) Resolution adaptive adjustment module: a sub-module predicts the "resolution evaluation value" of the input features, and then a second sub-module adaptively adjusts the original features based on this evaluation value. (2) Channel-spatial attention fusion module: global information of the channel and spatial dimensions is captured by performing global average pooling and global max pooling on the input features. (3) ASPP: multi-resolution contextual information is captured in parallel by using dilated convolutions with different dilation rates. (4) EMA: the extracted features are subjected to exponential weighted averaging to obtain a more stable and robust feature representation.

[0013] The prediction output module, based on the fused feature set, uses a negative partial log-likelihood loss function to predict the survival risk of GBM patients and optimize the training results. Specifically, the optimal frame is first determined using five-fold cross-validation on the training set TCGA-GBM; then, the frame parameters are frozen, and the frame is fine-tuned using three-fold cross-validation on the external queue ZN-GBM; finally, the fine-tuned frame is further optimized using three-fold cross-validation on the external queue CPTAC-GBM, resulting in a frame with good predictive performance in all three queues. The system outputs the C-index of the frame prediction score through cross-validation and external test queue validation.

[0014] The interpretive analysis module primarily outputs attention scores and risk scores. To enhance the transparency of the framework, this invention employs visualization techniques, such as generating attention activation maps to highlight key regions and performing t-SNE dimensional clustering to reveal patient stratification. Furthermore, by integrating multi-omics data, including spatiotemporal transcriptomics, high-throughput transcriptomics, and whole-exome sequencing, it delves into the biological mechanisms and pathways closely associated with risk indicators.

[0015] This application discloses a method for precise prognostic assessment of glioblastoma (GBM) patients. The system achieves precise prognostic assessment of GBM patients at the level of routine pathological sections. By combining clinical and risk indicators, it creates a nomogram that is easy for clinicians to use and understand. It possesses better consistency, sensitivity, specificity, and transparency, and can be deployed on cloud platforms and scanners, potentially significantly reducing the economic burden on patients and thus supporting personalized diagnosis and treatment of GBM patients.

[0016] This invention employs two visualization methods to achieve framework transparency, including drawing attention activation maps and t-SNE dimensional clustering, which improves the transparency of framework decisions.

[0017] This invention combines multi-omics data, including spatiotemporal transcriptomics, high-throughput transcriptomics, and whole-exome sequencing, overcoming the limitations of artificial intelligence in analyzing the complexity of the immune microenvironment.

[0018] The resolution adaptive adjustment module of the multi-resolution attention multi-bag inference framework involved in this invention preprocesses the input features to effectively address data differences at different resolutions.

[0019] The multi-resolution attention multi-bag inference framework involved in this invention, combined with the ASPP module, realizes multi-resolution feature extraction, thereby capturing rich information from fine cellular structure to overall tissue layout.

[0020] The multi-resolution attention multi-bag inference framework involved in this invention integrates a channel-space attention fusion module and an EMA module, which together constitute a powerful feature fusion mechanism. This effectively solves the information degradation effect that may be caused by traditional residual connections, and ensures the efficient transmission and integration of feature information in the network. Attached Figure Description

[0021] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0022] Figure 1Overall flowchart of accurate prognostic assessment for glioblastoma patients based on multi-resolution attention multi-bag reasoning framework;

[0023] Figure 2 Diagram of the multi-resolution attention multi-bag reasoning framework mechanism;

[0024] Figure 3 Attention activation map of a multi-resolution attention multi-bag reasoning framework;

[0025] Figure 4 t-SNE dimensional clustering graph of a multi-resolution attention multi-bag inference framework;

[0026] Figure 5 Nonograph of risk indicators combined with clinical indicators in a multi-resolution attention multi-baggage inference framework. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. The technical solutions provided by various embodiments of this invention will be described in detail below with reference to the accompanying drawings.

[0028] The following is a method for accurate prognostic assessment of glioblastoma patients involved in this invention. Figure 1 It includes the following sub-modules:

[0029] S1, Data Acquisition and Standardization Module;

[0030] S2, Image Preprocessing and Standardization Module;

[0031] S3, Multi-resolution attention multi-bag reasoning framework module;

[0032] S4, Prediction Output Module;

[0033] S5, Interpretive Analysis Module.

[0034] The data acquisition and standardization module includes the following inclusion criteria for GBM patients: patients must be at least 18 years old at the time of diagnosis; the GBM diagnosis must be confirmed by a qualified neuropathologist through both histological and molecular pathological examination according to the latest World Health Organization classification of central nervous system tumors; patients must be newly diagnosed and have not received any radiotherapy, chemotherapy, or targeted therapy; patients must have a Karnowski functional status score greater than 70; patients must have no history of other primary malignant tumors; patients must have complete clinical follow-up data, including treatment plans, imaging assessments, and survival outcomes, and their pathological tissue sections and paraffin blocks must be well preserved and usable for subsequent research; patient survival status includes death, survival, or censoring, with the starting point for calculating overall survival being the date of pathological diagnosis and the endpoint being the date of patient death or the date of the last follow-up visit.

[0035] The data acquisition and standardization module includes the following information: basic clinical information of GBM patients, covering age, sex, ethnicity, lesion location, medical history, treatment history, family history, and biomarker status (including IDH1 / 2, MGMT, TERT, EGFR, P53, Ki-67, etc.); follow-up data of patients, including the date of diagnosis, survival status, and last follow-up date; full-field digital pathological sections of pathological tissue after fixation, embedding, and staining, image data acquired using a high-magnification digital scanner, and image fragments extracted from them for subsequent framework training.

[0036] The image preprocessing and standardization module preprocesses digitized full-view slice images of GBM patients, mainly including: segmentation: removing useless background from the image; cutting: cutting the image into multiple non-overlapping 256×256 pixel image segments; feature extraction: using the pre-trained pathology-specific big language model UNI2-h to extract the local feature vector of each image patch, with a shape of (number of image patches, 1536), and using the pre-trained pathology-specific big language model Titan model to extract the global feature vector of the full-view image, with a shape of (1, 768). Then, the two feature sets are preprocessed and concatenated to form a comprehensive input feature set for use by subsequent modules.

[0037] Patients were followed up by telephone. The patients' survival status was death, survival, or censoring. The starting point for calculating the total survival was the date of the patient's pathological diagnosis, and the ending point was the date of the patient's death or the last follow-up. The follow-up information of GBM patients in the cohort was obtained by telephone. The pathological slides of GBM patients were digitized by scanning them at 40x magnification using a digital pathological slide scanner. The following steps were taken to preprocess the WSI data and extract the feature variables: (1) Load the WSI data and extract images at 20x magnification as needed; (2) Preprocess each image, including color normalization, noise reduction, contrast enhancement, etc.; (3) Divide the WSI into non-overlapping 256×256 image blocks; (4) Use a pre-trained pathological large model to extract features from each image block; (5) Map the extracted features onto a 1024×1024 feature vector. Furthermore, a multi-resolution attention multi-bag inference framework was designed and implemented. This framework can effectively capture key patterns and microscopic morphological changes in images, ranging from fine cellular structures to overall tissue layout. The specific model framework is as follows:

[0038] Step 1. Model Initialization:

[0039] 1.1 Initialize a multi-resolution attention multi-bag inference framework. 1.2 Define model hyperparameters, such as the number of layers, the number of attention heads, and the learning rate.

[0040] Step 2. Model Training and Optimization:

[0041] 2.1 The generated WSI-level features are used as input, along with corresponding clinical labels and inverse supervised learning for training. 2.2 The predictive performance is evaluated using a loss function (negative partial log-likelihood loss), and the model parameters are adjusted through backpropagation.

[0042] Step 3. Result Interpretation and Location:

[0043] 3.1 Based on the model's internal weights and gradient information, the Cox proportional hazards model is used to quantify the impact of each image patch on the final prediction result. 3.2 Based on these impact scores, a heatmap of the WSI or other visualizations are generated to reveal lesion areas and important structures.

[0044] Step 4. Testing and Evaluation:

[0045] 4.1 Evaluate the model using validation and test sets, and record the evaluation metrics for the prognostic prediction model, including the consistency index. 4.2 Analyze the model's performance on different datasets, compare it with the current state-of-the-art methods, and optimize the model.

[0046] This invention presents a multi-resolution attention-based multi-baggage inference framework focused on digital pathological slide analysis, designed to address the prognostic assessment problem for GBM patients. The framework incorporates modern artificial intelligence technologies such as resolution adaptive adjustment, channel-spatial attention fusion, ASPP, and EMA, and includes the following core modules (…). Figure 2 ):

[0047] Resolution Adaptive Adjustment Module: To overcome the feature inconsistency problem caused by different resolutions in pathological images, a resolution adaptive adjustment module is adopted. This module can intelligently detect the resolution of input features and adaptively adjust the features, thereby ensuring the robustness of the framework to multi-resolution images without introducing additional complexity. The proposed module can replace the traditional fixed feature extraction method, generate feature representations that are well adapted to different resolutions, and gracefully adjust as the input image resolution changes;

[0048] Channel-spatial attention fusion module: By adaptively weighting the channels and spatial dimensions of the feature map, the framework can dynamically focus on the feature channels that are more important to the current task, thereby emphasizing key information;

[0049] ASPP: By using multiple dilated convolutional layers with different dilation rates in parallel, along with a global average pooling layer, it aggregates features from different receptive fields. This design allows the framework to effectively expand its receptive field without increasing the number of parameters or sacrificing resolution, thereby capturing rich multi-resolution information from local details to global structure in images;

[0050] EMA: This smoothing technique helps reduce transient noise and fluctuations in features, thereby capturing more representative and generalizable feature information.

[0051] The specific implementation schemes for the resolution adaptive adjustment module, channel-spatial attention fusion module, ASPP, and EMA of the image preprocessing and normalization module are as follows:

[0052] 1. Resolution Adaptive Adjustment Module: Includes image block-level aggregation, magnification score prediction, feature adaptive transformation, and score modulation.

[0053] (1) Image patch-level aggregation, given the input image patch feature tensor ,in For batch size, Number of image blocks For the feature dimension. For the ... Each sample is pooled using mean pooling in the image patch dimension to obtain a bag-level vector. :

[0054] (1)

[0055] (2) Ratio score prediction, using a three-layer perceptron. Perform nonlinear mapping, For ReLU, It is a Sigmoid. Output Indicates the first Magnification fraction of each sample:

[0056] (2)

[0057] (3) Feature adaptive transformation, for each image patch vector Applying linear transformations, layer normalization, and activation yields the adapted features. .

[0058] (3)

[0059] (4) Scoring modulation, scalar Broadcasting is performed across image patches and channels, and the adapted features are amplitude modulated so that the feature intensity adaptively scales with magnification.

[0060] (4)

[0061] 2. Channel-Spatial Attention Fusion Module: Includes channel attention calculation, spatial attention calculation, and joint modulation.

[0062] (1) Channel attention calculation, given input The mean and max pooling operations are performed on the image patch dimension to obtain the results. and The channel weights are obtained by summing the two shared fully connected layers with ReLU and then passing them through a Sigmoid function. :

[0063] (5)

[0064] (2) Spatial attention calculation:

[0065] (6)

[0066] (3) Joint modulation, for each position First, perform mean and maximum aggregation on the channel dimension, stack the channels to form a two-channel time series, and then use a one-dimensional convolution with a kernel size of 7 and padding of 3. Convolution along the image patch dimension and then passing through a Sigmoid function to obtain the position weights. channel attention exist Dimensional broadcasting, spatial attention exist 3D broadcasting, multiplying the input features element-wise to obtain the jointly modulated features. :

[0067] (7)

[0068] 3. ASPP: Includes multi-resolution dilated convolution branch, image-level context branch, feature concatenation, fusion, and final feature map generation.

[0069] (1) Multi-resolution dilated convolution branch, given input For each expansion rate Perform dilated convolution on the branch (kernel size) (Step size 1, appropriate fill). Wherein express Convolution branch, the remaining branches are Hollow convolution yields :

[0070] (8)

[0071] (2) Image-level context branch: First, perform global average pooling on the entire feature map to obtain... ,through Convolutional mapping to After the channel, bilinear upsampling returns to ,get :

[0072] (9)

[0073] (3) Feature concatenation: The multi-resolution dilated convolution branch and the image-level context branch are concatenated in the channel dimension to form a pyramid-cascaded multi-resolution representation. :

[0074] (10)

[0075] (4) Fusion and generation of final feature maps, using Convolution performs linear fusion and dimensionality reduction on the concatenated channels to obtain the final multi-resolution context-enhanced features. :

[0076] (11)

[0077] 4. EMA: Includes exponentially weighted update, sliding feature fusion, time-dimensional cumulative smoothing, and final enhanced feature generation.

[0078] (1) Index-weighted update:

[0079] (12)

[0080] in Indicates the first Model parameters for each iteration This represents the smoothing parameter after EMA filtering. The decay coefficient determines the proportion of historical information retained. This update is performed after each iteration, continuously accumulating past weight information.

[0081] (2) Sliding feature fusion, given the current feature mapping Using the same EMA filtering method as the parameter update, a moving weighted average is applied to the features to smooth out instantaneous noise and improve feature stability.

[0082] (13)

[0083] (3) Time-dimensional cumulative smoothing: In order to further suppress fluctuations, in the most recent Accumulate the mean of the EMA features at each time step to obtain a more stable feature representation. :

[0084] (14)

[0085] (4) Finally, enhance the feature generation, and generate the smoothed features. Project back to the original channel dimension and compare with the features at the current time step. Adding them together yields the final enhanced feature map. .in The weights for the projective convolution are used to match dimensions and improve the fusion effect:

[0086] (15)

[0087] 5. The hyperparameter settings for framework training are as follows: During framework training, this experiment selects the Adam optimizer, sets the learning rate to 0.002, the batch size to 1, and the number of training epochs to 50.

[0088] 6. Model Output: After the framework training is complete, the fused feature information of each image segment can be output. This feature information contains rich image and tissue information. Then, through a decision-maker, the predicted risk indicators are obtained. Hazards are calculated, and the sigmoid function is used to map the risk indicators to the (0, 1) interval. Finally, the survival function is calculated using the cumulative product. In the survival analysis, we use the negative log-likelihood based on the Cox proportional hazards model as the loss function. It measures the difference between the risk indicators predicted by the framework and the actual observed survival time. The specific form of the loss function is as follows:

[0089] (16)

[0090] in, Indicates that an event has occurred (such as death); It is a risk set, contained in After or equal to All samples.

[0091] After the above training, the multi-resolution attentional multi-bag reasoning framework involved in this invention has achieved accurate prediction of the overall survival of GBM patients in the TCGA-GBM internal validation and CPTAC-GBM and ZN-GBM external validation cohorts (Table 1).

[0092] Table 1. The multi-resolution attention multi-bag reasoning framework accurately predicted the overall survival of GBM patients in the TCGA-GBM internal validation cohort and the CPTAC-GBM and ZN-GBM external validation cohorts.

[0093] TCGA-STAD CPTAC-STAD ZN-STAD C-Index (%) 76.98±0.018 76.53±0.054 71.73±0.021

[0094] 7. Framework Transparency Implementation: To achieve framework transparency, the multi-resolution attention multi-bag inference framework involved in this invention outputs attention scores after training. Using these output attention scores, we will improve model interpretability through two visualization methods: attention heatmaps and t-SNE dimensionality reduction clustering.

[0095] (1) Attention heatmap: The attention heatmap is used to visualize the contribution of different regions of the input image to the model output; for the preprocessed image patch, its attention score is expressed as:

[0096] (17)

[0097] Given an image patch / location feature set (No. The first sample Image patches, dimensions ,common (Each image patch) is first mapped to a scalar score using a small MLP or linear projection. Then, attention weights are normalized:

[0098] (18)

[0099] For the Perform a softmax operation on all scores for each sample to transform the original scores into a probability distribution. Attention weights are then assigned. Spatial mapping and upsampling are then performed.

[0100] (19)

[0101] vector Remodeling Image patch grid, then upsampling operator Expand it to the same pixel resolution as the input image. Resulting in pixel-level heatmaps (See the attention activation diagram for the multi-resolution attention multi-baggage reasoning framework) Figure 3 ).

[0102] (2) t-SNE dimensionality reduction clustering: t-SNE (t-distributed random neighborhood embedding) is a technique for dimensionality reduction of high-dimensional data. Its goal is to embed data points in a high-dimensional space into a low-dimensional space while maintaining the relative distance between data points. First, high-dimensional similarity is calculated:

[0103] (20)

[0104] (twenty one)

[0105] Given a high-dimensional dataset ,in It is the number of samples. It's about dimension. First, the points are calculated using the Gaussian distribution kernel function. and conditional probability between , The bandwidth parameter is adaptively determined based on perplexity. Then, the conditional probability is symmetric to obtain the similarity matrix in high-dimensional space. Then, low-dimensional similarity calculation is performed:

[0106] (twenty two)

[0107] Mapping high-dimensional data to a low-dimensional representation The value is typically 2 or 3. Low-dimensional similarity. A kernel function based on the Student-t distribution (degrees of freedom = 1) is used to calculate the kernel, giving the low-dimensional distribution a heavier tail, thus better preserving the local neighborhood structure. Then, Kullback–Leibler divergence is minimized.

[0108] (twenty three)

[0109] By minimizing the high-dimensional similarity distribution Similarity distribution with low dimension Optimize KL divergence During gradient descent, the gradient of the KL divergence is used to update the low-dimensional coordinates to maintain the mapping relationship of the high-dimensional neighborhood structure in the low-dimensional space as much as possible (see the t-SNE dimensional clustering diagram of the multi-resolution attention multi-bag inference framework). Figure 4 ).

[0110] 8. Exploration of Prognostic-Related Biological Processes: To explore the prognostic-related biological processes predicted by the framework, the multi-resolution attention-based multi-bag inference framework involved in this invention outputs attention scores and risk scores after training. Using the output risk indicators and transparency results, multi-omics analysis is performed in conjunction with spatiotemporal transcriptomics, high-throughput transcriptomics, and whole-exome sequencing.

[0111] (1) Combination of attention heatmap and spatiotemporal transcriptome: First, align the spot coordinates of the tissue slice image and the spatiotemporal transcriptome, and statistically analyze the gene set characteristics, cell composition and pathway activity of the high attention region; compare the high attention region of the model with the lesions or functional areas marked by the pathologist to evaluate the interpretability of the model.

[0112] (2) Combination of risk score and high-throughput transcriptomics: Perform differential expression analysis between high-risk and low-risk groups to see which genes or pathways are activated in high-risk populations; combine immune infiltration analysis, mutation burden, etc. to see if risk score is related to immune status and gene mutation patterns.

[0113] (3) Combination of risk score and whole exome: calculate the tumor burden score for each sample, compare the correlation between risk score and tumor burden score, and compare the gene mutation frequency of high and low risk groups.

[0114] 9. Model Integration with Clinical Indicators and Deployment: To achieve the deployment and clinical translation of the framework, the multi-resolution attention-based multi-baggage inference framework involved in this invention outputs an attention score after training. Using the output risk score, we construct a nomogram for estimating patient survival by combining it with clinical indicators (including age and gender) of GBM patients. This nomogram is then integrated into slide scanning equipment and a network platform for use by clinicians and patients (see the constructed nomogram for details). Figure 5 ).

[0115] The multi-resolution attentional multi-bag reasoning framework constructed in this invention provides an important tool for predicting overall survival in GBM patients. Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of this invention.

[0116] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention is also intended to include these modifications and variations. The above-described embodiments of this invention do not constitute a limitation on the scope of protection of this invention.

Claims

1. A method for implementing precise prognosis prediction of glioblastoma patients, characterized in that, The method is realized through modular process steps, including the following key modules: S1, data collection and standardization module, the data collection and standardization module defines the inclusion criteria of glioblastoma patients, collects the clinical basic data of patients, and performs digital conversion on pathological sections; S2, image preprocessing and standardization module, the image preprocessing and standardization module performs segmentation, segmentation and staining standardization operation on the digital image, to produce uniform image blocks for subsequent processing; then, the pre-trained pathological special disease large language model is used to extract the local feature vector of each image block, the shape is image block number times 1536, and the Titan model is used to extract the global feature vector of the whole field image, the shape is 1 times 768, then the two kinds of features are preprocessed and spliced to form a comprehensive input feature set for subsequent modules; S3, multi-resolution attention multi-bag reasoning framework module, the multi-resolution attention multi-bag reasoning framework module integrates local details and global semantics, and improves the representation quality with the help of feature integration components; this module integrates resolution adaptive adjustment, channel-space attention fusion, spatial conical pooling and exponential weighted smoothing mechanism, which can efficiently extract significant features and morphological variations from cell microtexture to tissue macrostructure; Among them, the resolution adaptive adjustment sub-module contains the following four processing steps: Step 1, image block level aggregation: （1） Given input image patch feature tensor where is the batch size, is the number of image patches, is the feature dimension, and is the mean-pooled bag-level vector for the th sample. Step 2, magnification score prediction: （2） with a three-layer perceptron for non-linear mapping, where is ReLU, is Sigmoid, and the output denotes the magnification score of the th sample. th sample. Step 3, feature adaptive transformation: （3） for each image patch vector applying a linear transformation with layer normalization and activation to obtain adapted features ; Step 4, score modulation: （4） Broadcasting on image block and channel dimensions, amplitude modulation is performed on the adapted features to make the feature intensity adaptively scale with the magnification factor. Broadcasting on image block and channel dimensions, amplitude modulation is performed on the adapted features to make the feature intensity adaptively scale with the magnification factor. S4, prediction output module, the prediction output module generates the integrated feature vector of each image block after the end of framework training, and then processes it through the classification decision maker to output the risk assessment index; S5, explanatory analysis module, the explanatory analysis module uses double visualization technology to enhance the transparency of the framework, including attention activation map and t-SNE dimension clustering; in addition, the explanatory analysis module further combines the risk indicators of the framework with the spatiotemporal transcriptome, high-throughput transcriptome and whole exome data for cross-omics joint analysis.

2. The method for realizing precise prognosis prediction of glioblastoma patients according to claim 1, characterized in that, The channel-space attention fusion sub-module contains the following three processing steps: Step 1, channel attention calculation: （5） Given input , mean and max pooling respectively on image patch dimension to get and , add after shared two fully connected and ReLU and get channel weight through Sigmoid ; Step 2, spatial attention calculation: （6） For each position First, mean and max pooling are done along the channel dimension, and the channels are stacked into two-channel temporal, then one-dimensional convolution with kernel size 7 and padding 3 Convolution along the image patch dimension and sigmoid to get position weight ; Step 3, joint modulation: （7） channel attention in spatial attention in spatial attention, element-wise multiplication of input features to get jointly modulated features .

3. The method for realizing precise prognosis prediction of glioblastoma patients according to claim 1, characterized in that, The operation of the spatial conical pooling sub-module contains the following four processing steps: Step 1, multi-resolution hollow convolution branch: （8） Given input For each expansion rate Perform dilated convolution on the branch, with a kernel size of [value missing]. The step size is 1, and appropriate padding is used, where express Convolution branch, the remaining branches are Hollow convolution yields ; Step 2, image-level context branch: （9） First, perform global average pooling on the entire feature map to obtain... ,through Convolutional mapping to After channeling, bilinear upsampling returns the shape to a height of H and a width of W, resulting in... ; Step 3, feature splicing: （10） The multi-resolution dilated convolution branch and the image-level context branch are concatenated in the channel dimension to form a pyramid-cascaded multi-resolution representation ; Step 4, fusion and generation of final feature mapping: （11） With Convolution is used to linearly fuse and reduce the dimension of the concatenated channels to obtain the final multi-resolution context enhanced features .

4. The method for realizing precise prognosis prediction of glioblastoma patients according to claim 1, characterized in that, The operation of the exponential weighted smoothing sub-module contains the following four processing steps: Step 1, exponential weighted update: （12） wherein denotes the model parameters of the step iteration, denotes the smoothed parameter filtered by the exponential moving average, is the decay coefficient, which determines the retention ratio of historical information. This update is performed after each iteration, continuously accumulating the weight information of the past. Step 2, sliding feature fusion: （13） Given the current feature map , the features are slidingly weighted and averaged to smooth out transient noise and improve feature stability using the same exponential moving average filtering method as the parameter update. Step 3, time dimension accumulation smoothing: （14） To further dampen the fluctuations, the exponential moving average features are mean-pooled over the last time steps, resulting in a more stationary feature representation ; Step 4, final enhanced feature generation: （15） Finally, the smoothed features Project back to the original channel dimension and compare with the features at the current time step. Adding them together yields the final enhanced feature map. ,in These are the weights for the projective convolution, used to match dimensions and improve fusion results.

5. The method for realizing precise prognosis prediction of glioblastoma patients according to claim 1, characterized in that, The risk indicators output by the framework are combined with the clinical parameters of glioblastoma patients to construct nomograms for estimating patient survival, and are integrated into section scanning equipment and network platforms for convenient access and application by clinical personnel and patients.

Images