Pathological recession grading system and method after neoadjuvant therapy of gastric cancer

Abstract

The invention provides a pathological recession grading system and method after neoadjuvant therapy of gastric cancer, and the system comprises a data input module which obtains a preoperative medical image and a postoperative pathological image of a target patient, a feature extraction module which extracts image omics and pathological omics features, a multi-modal fusion module which integrates the features to obtain a fusion vector, and a classification module which carries out classification on the fusion vector. The grading model module processes to obtain a result containing discrete grades, continuous scores and joint prediction, the interpretable module generates interpretable contents such as a regional heat map, and the clinical interface module integrates an evaluation report. The system improves the clinical credibility, and achieves the breakthrough of grading precision, efficiency and generalization ability.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a system and method for grading pathological regression after neoadjuvant therapy for gastric cancer. Background Art

[0002] Gastric cancer is a highly prevalent digestive tract malignancy, and neoadjuvant therapy (such as preoperative chemotherapy / chemoradiotherapy) has become an important treatment strategy for patients in the advanced and advanced stages. To ensure efficacy, treatment evaluation is necessary after treatment.

[0003] In current technology, tumor regression grade (TRG) is a key indicator guiding clinical decision-making in evaluating the efficacy of neoadjuvant cancer therapy. Traditional TRG grading relies on subjective interpretation of postoperative tissue sections by pathologists, which is time-consuming, highly subjective, and reliant on expert experience. Summary of the Invention

[0004] In view of this, the purpose of the present application is to provide a system and method for grading pathological regression after neoadjuvant therapy for gastric cancer to overcome the problems in the prior art.

[0005] In a first aspect, embodiments of the present application provide a system for grading pathological regression after neoadjuvant therapy for gastric cancer, the system comprising: A data input module is used to obtain preoperative medical images and postoperative pathological images of target patients; A feature extraction module, configured to extract radiomic features of the preoperative medical image and pathological features of the postoperative pathological image; A multimodal fusion module, for cross-modal interactive integration of the imaging omics features and the pathology omics features to obtain a fusion feature vector; A grading model module, configured to process the fused feature vector using a preset target grading model to obtain a prediction result for the target patient; wherein the prediction result includes discrete grade classification, continuous scoring, and joint prediction; An interpretable module, configured to generate interpretable content based on the prediction results; wherein the interpretable content includes a regional heat map, a feature contribution map, and a local causal path explanation; The clinical interface module is used to integrate the prediction results and the interpretable content into a clinically usable pathology assessment report for reference.

[0006] In some technical solutions of the present application, the data input module obtains the preoperative medical image and the postoperative pathological image in the following manner: The preoperative medical image is obtained by performing a first preprocessing on the preoperative initial image; wherein the first preprocessing includes region segmentation, density normalization, texture enhancement, and resampling; The postoperative pathological image is obtained by performing a second preprocessing on the postoperative initial image; wherein the second preprocessing includes: color standardization, background removal and tissue mask generation, image segmentation, and pathological structure enhancement.

[0007] In some technical solutions of the present application, the multimodal fusion module is used to interactively integrate the imaging genomics features and the pathology genomics features across modalities to obtain fusion features, including: Mapping the image feature embedding vector corresponding to the radiomics feature and the pathological feature embedding vector corresponding to the pathological feature to a unified dimensional space through linear projection; Using a disease attention head under a Transformer architecture, the pathological feature embedding vector is globally modeled to automatically learn adaptive attention weights for target type regions in the postoperative pathological image; Encoding the spatial coordinates of the preoperative medical image and the tissue position of the postoperative pathological image to generate a cross-modal position code; In the unified space, the multi-head cross attention mechanism of the Transformer architecture is utilized to interactively operate the image feature embedding vector and the pathological feature embedding vector according to the adaptive attention weight to obtain a fused feature vector.

[0008] In some technical solutions of the present application, the grading model module is used to process the fused feature vector using a preset target grading model to obtain a prediction result for the target patient, including: Performing high-order interactive refinement on the fused feature vector through a two-layer Transformer Encoder to obtain a semantically enhanced feature vector; The semantically enhanced feature vector is processed by adapting regression tasks and classification tasks through different output layers of MLP to obtain the discrete level classification, continuous scoring and joint prediction.

[0009] In some technical solutions of the present application, the above-mentioned hierarchical model module obtains the target hierarchical model by: After preprocessing and enhancing the training medical images and training case history images, they are input into the initial classification model and trained according to a preset training strategy, including warm-up and cosine annealing learning rate scheduling, mixed precision training to improve large-scale WSI processing efficiency, and cross-modal mutual information regularization. When a preset hybrid loss function meets a preset training requirement, the target classification model is obtained; wherein the hybrid loss function includes Focal Loss and Huber Loss.

[0010] In some technical solutions of the present application, the above-mentioned interpretable module is used to generate interpretable content based on the prediction results, including: generating the regional heat map according to the discrete level classification; generating the feature contribution graph according to the continuous score; The local causal path explanation is generated based on the joint prediction.

[0011] In some technical solutions of the present application, generating the regional heat map according to the discrete level classification includes: Based on discrete grade classification, pathology image heatmaps are generated through Grad-CAM++ and back-propagation gradients; Generating the feature contribution graph according to the continuous score includes: Based on the continuous score, the feature contribution is calculated and a preset number of features that affect the probability of withdrawal are displayed; Generating the local causal path explanation according to the joint prediction includes: Combining the ranks and probabilities of joint predictions, the causal paths of feature combinations are inferred with the help of structural causal models.

[0012] In a second aspect, an embodiment of the present application provides a system for grading pathological regression after neoadjuvant therapy for gastric cancer, the method comprising: Obtain preoperative medical images and postoperative pathological images of target patients through a data input module; Extracting radiomic features of the preoperative medical image and pathological features of the postoperative pathological image through a feature extraction module; Cross-modal interactive integration of the imaging omics features and the pathological omics features through a multimodal fusion module to obtain a fusion feature vector; Processing the fused feature vector using a preset target grading model through a grading model module to obtain a prediction result for the target patient; wherein the prediction result includes discrete grade classification, continuous scoring, and joint prediction; Generate explainable content based on the prediction results through an explainable module; wherein the explainable content includes a regional heat map, a feature contribution map, and a local causal path explanation; The prediction results and the explainable content are integrated into a clinically usable pathology assessment report through a clinical interface module for reference.

[0013] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for grading pathological regression after neoadjuvant therapy for gastric cancer when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned method for grading pathological regression after neoadjuvant therapy for gastric cancer are executed.

[0015] The technical solutions provided by the embodiments of the present application may have the following beneficial effects: The pathological regression grading system after neoadjuvant treatment of gastric cancer in the present application includes: a data input module for obtaining preoperative medical images and postoperative pathological images of the target patient; a feature extraction module for extracting the imaging genomics features of the preoperative medical images and the pathological genomics features of the postoperative pathological images; a multimodal fusion module for cross-modal interactive integration of the imaging genomics features and the pathological genomics features to obtain a fusion feature vector; a grading model module for processing the fusion feature vector using a preset target grading model to obtain a prediction result for the target patient; wherein the prediction result includes discrete grade classification, continuous scoring and joint prediction; an interpretable module for generating interpretable content based on the prediction result; wherein the interpretable content includes regional heat map, feature contribution map, and local causal path explanation; a clinical interface module for integrating the prediction result and the interpretable content into a clinically usable pathological assessment report for reference.

[0016] While improving clinical credibility, this application has achieved multi-dimensional breakthroughs in grading accuracy, efficiency, and generalization capabilities, providing a practical technical solution for the automation and precision of tumor efficacy evaluation.

[0017] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic diagram of a pathological regression grading system after neoadjuvant therapy for gastric cancer provided in an embodiment of the present application is shown; Figure 2 A schematic diagram of obtaining a fused feature vector provided in an embodiment of the present application is shown; Figure 3A flowchart of a pathological regression grading method after neoadjuvant therapy of gastric cancer is shown. Figure 4 A structural schematic diagram of an electronic device provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of description and illustration, and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts show the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowcharts or one or more operations can be removed from the flowcharts under the guidance of the content of the present application.

[0021] In addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0022] It should be noted that the term “comprising” will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0023] Gastric cancer is a high-incidence digestive tract malignant tumor, and its neoadjuvant therapy (such as preoperative chemotherapy / radiotherapy) has become an important treatment strategy for advanced cases. After treatment, in order to ensure the effect, the treatment effect needs to be evaluated.

[0024] In the prior art, in the evaluation of the effect after neoadjuvant therapy of tumors, tumor regression grading (TRG) is a key indicator for guiding clinical decision-making. The current traditional TRG grading is subjectively judged by pathologists based on the amount of residual cancer cells under a high-power microscope. This method has the following limitations: strong subjectivity: poor consistency between different pathologists (κ value is often <0.5); time-consuming and difficult to process in batches: multiple slice areas need to be interpreted field by field; cannot be continuously quantified: only grading, lack of refinement; easy to miss some small residual foci, especially mucous gland cancer, residual microfocus type, etc.

[0025] Based on this, the application embodiment provides a gastric cancer neoadjuvant therapy post-pathological regression grading system and method, which is described below. Some embodiments of the application are described in detail below. In the case of no conflict, the embodiments described below and the features in the embodiments can be combined with each other.

[0026] Figure 1 A schematic diagram of a gastric cancer neoadjuvant therapy post-pathological regression grading system provided by an embodiment of the application is shown, and the system comprises: A data input module for obtaining preoperative medical images and postoperative pathological images of a target patient; A feature extraction module for extracting radiomic features of the preoperative medical images and pathomic features of the postoperative pathological images; A multi-modal fusion module for cross-modal interaction and integration of the radiomic features and the pathomic features to obtain a fusion feature vector; A grading model module for processing the fusion feature vector using a pre-set target grading model to obtain a prediction result of the target patient; wherein the prediction result includes discrete level classification, continuous score and joint prediction; An interpretable module for generating interpretability content according to the prediction result; wherein the interpretability content includes region heat map, feature contribution map, and local causal path explanation; A clinical interface module for integrating the prediction result and the interpretability content into a clinically usable pathological evaluation report for reference.

[0027] When the data input module obtains image data, the first obtained is the preoperative initial image and the postoperative initial image. The preoperative initial image can be the image of the CT or MRI abdominal scan of the target patient before the operation, in the original DICOM format. The postoperative initial image can be the H&E full field scanning (WSI) of the full cut specimen after the operation, and the image after 40x magnification.

[0028] After obtaining the preoperative initial image and the postoperative initial image, in order to ensure accuracy and robustness, the preoperative initial image and the postoperative initial image need to be preprocessed. Since the preprocessing processes of the two are different, in order to distinguish them, the preprocessing of the preoperative initial image is called first preprocessing, and the preprocessing of the postoperative initial image is called second preprocessing. The first preprocessing includes region segmentation, density standardization, texture enhancement, resampling, etc., and the second preprocessing includes color standardization, background removal and tissue mask generation, image segmentation, and pathological structure enhancement.

[0029] Specifically, the first preprocessing of the preoperative initial image includes: Regional segmentation (precise tumor ROI segmentation) is implemented by combining the 3D-Slicer semi-automatic segmentation algorithm with manual correction by pathologists to precisely delineate the tumor region, ensuring that the ROI includes the infiltrated portion of the boundary (such as the transition zone between the tumor and the normal gastric wall). Normal tissue interference is avoided, for example, by excluding perigastric adipose tissue from the ROI.

[0030] Density normalization is implemented by linearly stretching the HU values ​​of CT images within the range of -1000 to 400 to achieve uniform soft tissue density distribution. This enhances the contrast between post-radiation fibrosis (CT values ​​20-60 HU) and necrosis (-20-10 HU). For example, the standard deviation of CT values ​​in fibrotic areas can be reduced from 35 HU to 15 HU, improving the stability of subsequent texture features (such as GLRLM long-run emphasis).

[0031] Core texture enhancement algorithm: CLAHE (Contrast Limited Adaptive Histogram Equalization) is applied to locally enhance fiber texture. This highlights fiber structural details, helps capture microfibrosis areas after radiotherapy, and more accurately reflects tissue heterogeneity.

[0032] Resampling (isotropic) protocol: CT / MRI images of varying slice thicknesses are uniformly resampled to 1.0 mm³ isotropic voxels to ensure consistency across devices.

[0033] Second pre-processing of the initial postoperative image: Color normalization: The color distribution of H&E-stained WSI was matched to that of reference slides (e.g., mean values ​​of hematoxylin staining [100, 50, 60], mean values ​​of eosin staining [200, 120, 80]) using the Reinhard normalization method.

[0034] Background removal and tissue mask generation: Through RGB threshold segmentation (such as pixel value > 230 is judged as blank) combined with morphological operations, blank areas and fat tissue (vacuolar effect) are removed.

[0035] Image segmentation (sliding window): Patch-level processing: A 256×256 pixel sliding window (step size 128 pixels) was used to segment the WSI, and only patches with tissue coverage ≥80% were retained.

[0036] Pathological structure enhancement, targeting the following scenarios: Mucinous adenocarcinoma: Applying LoG (Laplacian of Gaussian) filtering to highlight the boundaries of the mucus pool, improving the gradient characteristics of the mucus matrix by 30%; Ulcerative carcinoma: Using a local contrast enhancement algorithm to strengthen the boundaries between necrotic areas and surviving cancer cells, facilitating subsequent recognition by the attention mechanism.

[0037] After obtaining preoperative medical images and postoperative pathological images, feature extraction is required to obtain radiomic features of the preoperative medical images and pathological features of the postoperative pathological images. The feature extraction module transforms raw medical images into semantic features through quantitative radiomics analysis and deep learning feature extraction using pathological omics, providing core data support for multimodal fusion and accurate TRG grading.

[0038] Radiomics feature extraction from preoperative medical images: Preoperative CT / MRI images after initial preprocessing (ROI segmentation and density normalization) were extracted using the pyradiomics open-source library, with approximately 120 dimensions of high-order texture, shape, and statistical features. Texture features, including entropy, contrast, and correlation of the gray-level co-occurrence matrix (GLCM), reflect tumor heterogeneity. Shape features, including sphericity and compactness, quantify tumor morphological regularity. Statistical features, including the mean and variance of the gray-level histogram, characterize density distribution. High-order features, including long-run emphasis of the gray-level run-length matrix (GLRLM), capture fiber structural arrangement patterns. Redundancy filtering: highly correlated features were removed through Pearson correlation analysis (threshold 0.9), for example, when the correlation between GLCM entropy and GLRLM long run emphasis was greater than 0.8, only one of them was retained; dimension optimization: the original 132 features were filtered to 128 valid features, for example, key features such as GLCM_Entropy and Shape_Compactness were retained to reduce computational complexity.

[0039] Pathomics feature extraction from postoperative pathology images: A deep learning-based feature extraction architecture is used to extract pathology patches (256×256 pixels, ≥80% tissue coverage) after secondary preprocessing. ResNet50 is pretrained (ImageNet pretrained) to extract 2048-dimensional semantic features. Multi-scale CNN (MS-CNN) or Vision Transformer (ViT) are optional to capture pathological structures at different scales (e.g., single cell nuclei to cell nests). Single-patch feature extraction: The average pooling layer of ResNet50 outputs a 2048-dimensional vector representing structural features such as nuclear atypia and fibrosis arrangement. Full-slice feature aggregation: Using an attention mechanism, all valid patch features are weighted and aggregated, assigning higher weights to "highly suspected residual regions" (such as basal cancer cell islands) to generate a 512-dimensional pathomic feature vector. Pathological structure-specific optimization is also employed. For mucinous adenocarcinoma, grayscale gradient features are additionally extracted from patches in mucin pools to enhance mucin matrix recognition. For microfocal residuals, features are extracted from cancer lesions <500μm in diameter using super-resolution reconstruction to 512×512 pixels, preserving subcellular details (such as chromatin distribution).

[0040] After obtaining the radiomics features and pathological features, it is necessary to use the multimodal fusion module to perform cross-modal processing on the two to obtain a fused feature vector. The multimodal fusion module achieves deep interaction between radiomics and pathological features through dimensional unification, spatial encoding, and the coordination of attention mechanisms, providing a fused feature vector containing cross-modal semantics for subsequent TRG grading, significantly improving the model's comprehensive evaluation ability for treatment response. Specifically, Figure 2 As shown, the fusion is performed according to the following steps: S201, mapping the image feature embedding vector corresponding to the radiomics feature and the pathology feature embedding vector corresponding to the pathology feature to a unified dimensional space through linear projection; S202, using a disease attention head under a Transformer architecture to globally model the pathological feature embedding vector, and automatically learn adaptive attention weights for target type regions in the postoperative pathological image; S203, encoding the spatial coordinates of the preoperative medical image and the tissue position of the postoperative pathological image to generate a cross-modal position code; S204. In the unified space, using the multi-head cross attention mechanism of the Transformer architecture, interactively operate the image feature embedding vector and the pathology feature embedding vector according to the adaptive attention weight to obtain a fused feature vector.

[0041] For example, after obtaining the radiomics features (128 dimensions) of preoperative medical images and the pathological features (512 dimensions) of postoperative pathological images, each modality is processed by a separate Encoder to convert the radiomics features and pathological features into feature embedding vectors; then, the disease attention head is introduced using the cross-modal interactive Transformer. This attention sub-module tends to focus on the representation interaction of high-probability areas of residual cancer cells, radiotherapy fibrosis boundary areas, and necrotic areas by learning training data; at the same time, cross-modal position encoding is added to encode the correspondence between the CT image position and the slice tissue space and integrate it into the feature interaction process; finally, the processed image is interacted with the pathological feature embedding vector, and feature fusion is achieved through a multi-head cross-attention mechanism. The vector dimension after modal fusion is uniformly set to 768 dimensions, thereby obtaining a fused feature vector. This vector contains the key information of the two modalities and their interaction relationship, providing data support for subsequent hierarchical prediction.

[0042] After obtaining the fused feature vector, the fused feature vector is processed using a hierarchical model module to obtain a prediction result for the target patient; wherein the prediction result includes discrete grade classification, continuous scoring, and joint prediction.

[0043] The grading model module first uses the preset target grading model to process the fused feature vector, and then uses two layers of Transformer Encoder to perform high-order interactive extraction on the fused feature vector, capturing the deep correlation between features to obtain a semantically enhanced feature vector. Then, the semantically enhanced feature vector is processed by adapting the regression and classification tasks through different output layers of the MLP. The classification task outputs discrete grade classification results, and the regression task generates continuous scores. Finally, the two are combined to achieve joint prediction, thereby obtaining a complete prediction result for the target patient including discrete grades, continuous scores and joint predictions, providing multi-dimensional quantitative indicators for clinical evaluation.

[0044] Among them, the hierarchical model module mainly uses the target hierarchical model, which is trained in the following way: after preprocessing and enhancing the training medical images and training medical record images, they are input into the initial hierarchical model, and the initial hierarchical model is trained according to the preset training strategy; wherein, the training strategy includes Warm-up and Cosine Annealing learning rate scheduling, mixed precision training to improve large-scale WSI processing efficiency and cross-modal mutual information regularization; when the preset mixed loss function meets the preset training requirements, the target hierarchical model is obtained; wherein, the mixed loss function includes Focal Loss and Huber Loss.

[0045] Specifically, the training medical images and training pathology images were preprocessed and enhanced, including multi-pathology expert labeling (each WSI was independently labeled with AI-TRG grade by at least two pathology directors. If there was any inconsistency, a three-review decision was made to ensure the accuracy of the training labels), patch-level labeling (highlighting key residual cancer areas (local pCR residual points, fibrosis and necrosis boundaries) for attention-guided training), pathology block rotation, hue perturbation, elastic deformation, CT image mirror flipping, affine transformation and other data enhancement operations. The pseudo-label self-distillation strategy was used for weakly labeled samples; the processed data was input into the initial grading model and trained according to the preset training strategy, which included warm-up learning rate from 10⁻ 6 Increased to 5×10⁻ 4 Cosine Annealing scheduling is then performed, FP16 mixed precision training is used to improve the efficiency of large-scale WSI processing, and cross-modal mutual information regularization is used to ensure that the image and pathological characteristics change synchronously. During the training process, a hybrid loss function including Focal Loss (for handling classification tasks and enhancing the recognition of minority categories) and Huber Loss (for stabilizing continuous scores) is used, and the loss ratio is dynamically adjusted according to the category distribution during training. When the hybrid loss function meets the preset training requirements, a target grading model that can output discrete grade classification, continuous scoring and joint prediction is obtained.

[0046] The interpretable module generates interpretable content based on the prediction results output by the hierarchical model. The specific process is as follows: based on the discrete grade classification results, the Grad-CAM++ algorithm is combined with back-propagation gradients to generate a pathology image heat map, and after high-resolution upsampling, key areas such as residual cancer foci and fibrosis are highlighted in the pathology image; based on the continuous score, the SHAP value is used to calculate the marginal contribution of each feature to the regression probability, and after sorting, a preset number of core features (such as GLCM Entropy, tumor shape characteristics, etc.) and their quantitative impact on the score are displayed; combined with the joint predicted grade and probability, the structural causal model is used to infer the causal logic chain of the feature combination, for example, the causal path from "preoperative image texture disorder + postoperative pathological fibrosis degree" to the specific TRG grade determination is deduced, thereby comprehensively presenting the model decision-making basis.

[0047] The clinical interface module is responsible for integrating the evaluation results of the system into a clinically usable pathological evaluation report. The specific process is as follows: the regression grade output by the grading model (such as AI-TRG 0-3 grade), the confidence score (a probability value between 0 and 1), and the explainability content generated by the explainable module, such as regional heat map, feature contribution map, and local causal path explanation, are integrated to automatically generate a structured report. The report includes AI-TRG grade, confidence score, heat map superimposed on original pathology graph (supports transparency adjustment), feature contribution Top list (in chart form), and key reasoning area screenshot. At the same time, a pathology physician review terminal is provided to support manual review, confirmation, and feedback modification. The feedback information can be used for system incremental learning. In addition, the module supports HL7 / FHIR protocol, can be seamlessly integrated with hospital LIS and EMR systems, and adopts Docker containerized deployment. Through the development of API interfaces based on FastAPI standard, cross-platform compatibility is ensured. Ultimately, the pathology physician is provided with intuitive and comprehensive clinical reference information to assist in treatment effect evaluation and decision-making.

[0048] In the data input module of the embodiments of the present application, the heterogeneity of image and pathology data is eliminated through preprocessing, so that the feature extraction module can accurately obtain 128-dimensional imageomics features and 512-dimensional pathologyomics features, providing high-quality input for the multi-modal fusion module. The multi-modal fusion module maps the cross-modal features to a 768-dimensional unified space with the help of Transformer and disease attention head, captures deep correlations such as "image texture disorder-pathological fibrosis", and promotes the grading model module to extract high-order features through two layers of Transformer Encoder, combined with the mixed loss function of Focal Loss and Huber Loss, which improves the TRG four-classification accuracy and small focus cancer residual detection rate. The explainable module generates regional heat maps and SHAP feature contribution maps based on the prediction results, which is convenient for clinicians to understand and receive. The clinical interface module integrates the evaluation results into a standard structured report, which reduces the evaluation time and realizes the efficiency and accuracy improvement of the whole chain from data standardization to clinical decision-making.

[0049] In an optional embodiment, the basic information is as follows: patient gender: male, age: 62 years old; Preoperative clinical stage: T3N2M0; preoperative treatment plan: SOX neoadjuvant chemotherapy for 4 cycles; specimen processing: total gastrectomy + systematic lymph node dissection, H&E staining of the whole section digital scanning; supplementary data: preoperative abdominal enhanced CT scan.

[0050] System execution process: 1. Radiomics processing: semi-automatic segmentation of the ROI tumor area and expert calibration; radiomics feature extraction (132 texture, shape, and grayscale features); feature normalization and redundant filtering (Pearson correlation threshold 0.9) to retain 128 valid features.

[0051] 2. Pathology image processing: WSI color standardization and tissue mask generation; patch extraction (256×256), retaining approximately 9,800 valid tissue patches; special attention to boundary enhancement of ulcerative areas.

[0052] 3. Feature fusion and graded prediction: Multimodal Transformer fuses image and pathological features; outputs AI-TRG = 1 (partial pathological regression); and simultaneously determines it as non-complete response (non-pCR) with a confidence level of 87.2%.

[0053] 4. Interpretable output: Generate Grad-CAM heatmap of residual cancer lesions; Top radiomics features: GLCM Entropy, Shape Compactness; Output structured assessment report to the pathology system.

[0054] 5. Clinical review results: The pathologist's review was completely consistent with the AI-TRG prediction; review time was reduced by 65%, improving diagnostic efficiency.

[0055] Figure 3 A flow chart of a method for grading pathological regression after neoadjuvant therapy for gastric cancer provided in an embodiment of the present application is shown, wherein the method includes steps S101-S106; specifically: S101, obtaining preoperative medical images and postoperative pathological images of a target patient through a data input module; S102, extracting radiomic features of the preoperative medical image and pathological features of the postoperative pathological image through a feature extraction module; S103, integrating the imaging omics features and the pathology omics features through a multimodal fusion module to obtain a fusion feature vector; S104. Processing the fused feature vector using a preset target grading model through a grading model module to obtain a prediction result for the target patient; wherein the prediction result includes discrete grade classification, continuous scoring, and joint prediction; S105. Generate explainable content based on the prediction results through an explainable module; wherein the explainable content includes a regional heat map, a feature contribution map, and a local causal path explanation; S106. Integrate the prediction results and the explainable content into a clinically usable pathology assessment report through a clinical interface module for reference.

[0056] The preoperative medical images and the postoperative pathological images are obtained by: The preoperative medical image is obtained by performing a first preprocessing on the preoperative initial image; wherein the first preprocessing includes region segmentation, density normalization, texture enhancement, and resampling; The postoperative pathological image is obtained by performing a second preprocessing on the postoperative initial image; wherein the second preprocessing includes: color standardization, background removal and tissue mask generation, image segmentation, and pathological structure enhancement.

[0057] The multimodal fusion module integrates the imaging omics features and the pathology omics features across modalities to obtain fusion features, including: Mapping the image feature embedding vector corresponding to the radiomics feature and the pathological feature embedding vector corresponding to the pathological feature to a unified dimensional space through linear projection; Using a disease attention head under a Transformer architecture, the pathological feature embedding vector is globally modeled to automatically learn adaptive attention weights for target type regions in the postoperative pathological image; Encoding the spatial coordinates of the preoperative medical image and the tissue position of the postoperative pathological image to generate a cross-modal position code; In the unified space, the multi-head cross attention mechanism of the Transformer architecture is utilized to interactively operate the image feature embedding vector and the pathological feature embedding vector according to the adaptive attention weight to obtain a fused feature vector.

[0058] The fused feature vector is processed using a preset target classification model to obtain a prediction result for the target patient, including: Performing high-order interactive refinement on the fused feature vector through a two-layer Transformer Encoder to obtain a semantically enhanced feature vector; The semantically enhanced feature vector is processed by adapting regression tasks and classification tasks through different output layers of MLP to obtain the discrete level classification, continuous scoring and joint prediction.

[0059] The target classification model is obtained by: After preprocessing and enhancing the training medical images and training case history images, they are input into the initial classification model and trained according to a preset training strategy, including warm-up and cosine annealing learning rate scheduling, mixed precision training to improve large-scale WSI processing efficiency, and cross-modal mutual information regularization. The target hierarchical model is obtained when a preset mixed loss function meets a preset training requirement, wherein the mixed loss function comprises a Focal Loss and a Huber Loss.

[0060] The explainability content is generated according to the prediction result, including: The region heat map is generated according to the discrete hierarchical classification. The feature contribution map is generated according to the continuous score. The local causal path explanation is generated according to the joint prediction.

[0061] The region heat map is generated according to the discrete hierarchical classification, including: The pathological image heat map is generated by Grad-CAM++ and back propagation gradient based on the discrete hierarchical classification. The feature contribution map is generated according to the continuous score, including: The feature contribution is calculated and a preset number of features affecting the retreat probability are displayed according to the continuous score. The local causal path explanation is generated according to the joint prediction, including: The causal path of the feature combination is inferred by means of a structural causal model in combination with the grade and probability of the joint prediction.

[0062] As shown in Figure 4 The embodiment of the present application provides an electronic device for executing the gastric cancer post-neoadjuvant therapy pathological regression grading method in the present application. The device comprises a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to realize the steps of the gastric cancer post-neoadjuvant therapy pathological regression grading method.

[0063] Specifically, the memory and the processor can be general memory and processor, which are not limited here. When the processor runs the computer program stored in the memory, the gastric cancer post-neoadjuvant therapy pathological regression grading method can be executed.

[0064] Corresponding to the gastric cancer post-neoadjuvant therapy pathological regression grading method in the present application, the embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by the processor to execute the steps of the gastric cancer post-neoadjuvant therapy pathological regression grading method.

[0065] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. The computer program on the storage medium is executed to execute the gastric cancer post-neoadjuvant therapy pathological regression grading method.

[0066] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, indirect coupling or communication connection of the system or unit, which can be electrical, mechanical or other forms.

[0067] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment as needed.

[0068] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0069] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0070] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0071] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. However, these modifications, changes, or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims.

Claims

1. A grading system for pathological regression after neoadjuvant therapy for gastric cancer, characterized in that: The system comprises: A data input module is used to obtain preoperative medical images and postoperative pathological images of target patients; A feature extraction module, configured to extract radiomic features of the preoperative medical image and pathological features of the postoperative pathological image; A multimodal fusion module, for cross-modal interactive integration of the imaging omics features and the pathology omics features to obtain a fusion feature vector; A grading model module, configured to process the fused feature vector using a preset target grading model to obtain a prediction result for the target patient; wherein the prediction result includes discrete grade classification, continuous scoring, and joint prediction; An interpretable module, configured to generate interpretable content based on the prediction results; wherein the interpretable content includes a regional heat map, a feature contribution map, and a local causal path explanation; The clinical interface module is used to integrate the prediction results and the interpretable content into a clinically usable pathology assessment report for reference.

2. The system according to claim 1, wherein: The data input module obtains the preoperative medical image and the postoperative pathological image in the following manner: The preoperative medical image is obtained by performing a first preprocessing on the preoperative initial image; wherein the first preprocessing includes region segmentation, density normalization, texture enhancement, and resampling; The postoperative pathological image is obtained by performing a second preprocessing on the postoperative initial image; wherein the second preprocessing includes: color standardization, background removal and tissue mask generation, image segmentation, and pathological structure enhancement.

3. The system according to claim 1, wherein: The multimodal fusion module is used to interactively integrate the imaging genomics features and the pathological genomics features across modalities to obtain fusion features, including: Mapping the image feature embedding vector corresponding to the radiomics feature and the pathological feature embedding vector corresponding to the pathological feature to a unified dimensional space through linear projection; Using a disease attention head under a Transformer architecture, the pathological feature embedding vector is globally modeled to automatically learn adaptive attention weights for target type regions in the postoperative pathological image; Encoding the spatial coordinates of the preoperative medical image and the tissue position of the postoperative pathological image to generate a cross-modal position code; In the unified space, the multi-head cross attention mechanism of the Transformer architecture is utilized to interactively operate the image feature embedding vector and the pathological feature embedding vector according to the adaptive attention weight to obtain a fused feature vector.

4. The system according to claim 1, wherein: The grading model module is used to process the fused feature vector using a preset target grading model to obtain a prediction result for the target patient, including: Performing high-order interactive refinement on the fused feature vector through a two-layer Transformer Encoder to obtain a semantically enhanced feature vector; The semantically enhanced feature vector is processed by adapting regression tasks and classification tasks through different output layers of MLP to obtain the discrete level classification, continuous scoring and joint prediction.

5. The system according to claim 1, wherein: The hierarchical model module obtains the target hierarchical model by: After preprocessing and enhancing the training medical images and training case history images, they are input into the initial classification model and trained according to a preset training strategy, including warm-up and cosine annealing learning rate scheduling, mixed precision training to improve large-scale WSI processing efficiency, and cross-modal mutual information regularization. When a preset hybrid loss function meets a preset training requirement, the target classification model is obtained; wherein the hybrid loss function includes Focal Loss and Huber Loss.

6. The system according to claim 1, wherein: The interpretable module is used to generate interpretable content according to the prediction result, including: generating the regional heat map according to the discrete level classification; generating the feature contribution graph according to the continuous score; The local causal path explanation is generated based on the joint prediction.

7. The system according to claim 6, characterized in that Generating the regional heat map according to the discrete level classification includes: Based on discrete grade classification, pathology image heatmaps are generated through Grad-CAM++ and back-propagation gradients; Generating the feature contribution graph according to the continuous score includes: Based on the continuous score, the feature contribution is calculated and a preset number of features that affect the probability of withdrawal are displayed; Generating the local causal path explanation according to the joint prediction includes: Combining the ranks and probabilities of joint predictions, the causal paths of feature combinations are inferred with the help of structural causal models.

8. A method for grading pathological regression after neoadjuvant therapy for gastric cancer, characterized in that: The method comprises: Obtain preoperative medical images and postoperative pathological images of target patients through a data input module; Extracting radiomic features of the preoperative medical image and pathological features of the postoperative pathological image through a feature extraction module; Cross-modal interactive integration of the imaging omics features and the pathological omics features through a multimodal fusion module to obtain a fusion feature vector; Processing the fused feature vector using a preset target grading model through a grading model module to obtain a prediction result for the target patient; wherein the prediction result includes discrete grade classification, continuous scoring, and joint prediction; Generate explainable content based on the prediction results through an explainable module; wherein the explainable content includes a regional heat map, a feature contribution map, and a local causal path explanation; The prediction results and the explainable content are integrated into a clinically usable pathology assessment report through a clinical interface module for reference.

9. An electronic device, characterized in that: include: A processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of the method for grading pathological regression after neoadjuvant treatment of gastric cancer are performed as described in claim 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the method for grading pathological regression after neoadjuvant therapy for gastric cancer as claimed in claim 8.

Images