Colorectal cancer prognosis analysis system and colorectal cancer prognosis analysis method fusing tissue and block characteristics

Abstract

The invention relates to the technical field of pathological image processing and medical artificial intelligence auxiliary diagnosis, and discloses a colorectal cancer prognosis analysis system and method fusing tissue and block characteristics, and the system comprises an image block cutting and standardization module which generates a standardized image block set under different multiplying power; the ResNet50 classifier module is used for carrying out organization classification on the image blocks; the tissue type specific feature extraction module is used for extracting multi-scale survival correlation features; the tissue spatial feature extraction and quantitative analysis module is used for reconstructing a multi-tissue type distribution diagram and calculating quantitative spatial features such as the number of lymphocytes and the intraperitoneal ratio after morphological repair; and the multi-modal feature fusion and survival prediction module is used for fusing the pathological deep features, the spatial segmentation features and the clinical data, and outputting an individualized prognosis risk score by using a survival model. According to the method, a macrostructure and microcosmic details are effectively integrated, and the accuracy and interpretability of colorectal cancer survival risk layering are improved.

Description

Technical Field

[0001] This invention relates to the fields of pathological image processing and medical artificial intelligence-assisted diagnosis technology, specifically to a prognostic analysis system and method for colorectal cancer that integrates tissue and patch features. Background Technology

[0002] Colorectal cancer is one of the leading causes of cancer death worldwide. Despite continuous advancements in medical techniques, patient survival outcomes remain significantly different due to the complexity of tumor biological behavior and the high heterogeneity of its microenvironment.

[0003] Currently, clinical prognostic assessment mainly relies on the TNM staging system and histopathological grading. However, these traditional methods have limitations in revealing tumor heterogeneity and often fail to meet the needs of personalized precision treatment. Although molecular and multi-omics analyses can provide in-depth biological information, their high cost and long testing cycles limit their widespread application in routine clinical practice.

[0004] Whole-section images of histopathological tissue contain detailed morphological and spatial distribution information of various tissue components such as tumor epithelium, lymphocytes, stroma, and mucus area, and are an important basis for prognostic assessment.

[0005] However, traditional manual slide interpretation relies heavily on the pathologist's personal experience, which is not only labor-intensive but also suffers from strong subjectivity and poor inter-observer consistency, making it difficult to objectively and quantitatively analyze the complex spatial tissue structures in whole slides. In recent years, deep learning-based artificial intelligence technology has made progress in the field of pathological image analysis, such as using convolutional neural networks for tissue classification or feature extraction. However, existing deep learning analysis methods still face technical bottlenecks.

[0006] On the one hand, while some existing technologies utilize deep learning to extract image features, they are often limited to image information at a single scale, failing to simultaneously take into account the macroscopic architecture of tissues and the microscopic morphological details of cells, and lacking effective fusion of multimodal data (such as pathological features, spatial structural features and clinical baseline data), which limits the accuracy and robustness of prognostic prediction.

[0007] On the other hand, existing end-to-end deep learning models usually lack the ability to quantitatively represent the spatial topology of the tumor microenvironment, making it difficult to accurately calculate structured indicators with clear biological significance, such as tumor immune cell interactions and matrix reactions. Moreover, the model decision-making process is often a black box, lacking sufficient interpretability and making it difficult to intuitively display the pathological areas that the model focuses on, thus limiting its trustworthiness and application value in clinical auxiliary diagnosis. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides a colorectal cancer prognostic analysis system and method that integrates tissue and patch features. This solves the problems of existing technologies being limited to single-scale image information, lacking effective multimodal fusion of pathological features, spatial structure and clinical data, and being difficult to quantitatively characterize the spatial topology of the tumor microenvironment and lacking model interpretability.

[0009] The first aspect of this invention provides a colorectal cancer prognostic analysis system that integrates tissue and patch features. The system mainly includes an image patch segmentation and standardization module, a ResNet50 classifier module, a tissue type-specific feature extraction module, a tissue spatial feature extraction and quantitative analysis module, and a multimodal feature fusion and survival prediction module.

[0010] The image patch segmentation and normalization module is used to parse the input whole-slice image of colorectal cancer pathology, select effective regions from the whole-slice image of colorectal cancer pathology through a random sampling strategy, generate normalized image patch sets at different magnifications, and convert the normalized image patch sets into Tensor format.

[0011] The ResNet50 classifier module is used to encode features of the preprocessed and converted to Tensor format normalized image patch set, and output the predicted probabilities and tissue classification results of nine types of pathological tissues.

[0012] The tissue type-specific feature extraction module is used to extract the standardized image patch set at different magnifications in parallel, and to generate a patient-level comprehensive feature matrix using a feature extraction network trained with patient survival data as a supervision signal.

[0013] The tissue spatial feature extraction and quantitative analysis module is used to map the standardized image patch set back to the full slice coordinate space based on the tissue classification results to reconstruct a multi-tissue type distribution map, and to perform morphological repair and connected component analysis on the multi-tissue type distribution map to generate a segmentation feature matrix.

[0014] The multimodal feature fusion and survival prediction module is used to fuse the comprehensive feature matrix, the segmentation feature matrix, and the patient's numerical clinical data to generate a complete multimodal feature matrix, and to output an individualized prognostic risk score using a survival model.

[0015] Furthermore, the image patch cutting and normalization module includes a Patch-Extract cutting unit and a Norm-Transform normalization unit.

[0016] The Patch-Extract cutting unit parses the head information of the full slice image to obtain the original magnification and calculates the scaling factor. It adopts a random sampling strategy based on tissue region detection, selects effective tissue regions from the full slice according to a preset ratio, and extracts the standardized image patch set. The different magnifications include at least 10x magnification and 40x magnification.

[0017] The Norm-Transform normalization unit receives the normalized image patch set and converts it into Tensor format, converts the pixel data into floating-point tensors, and uses the normalization formula to correct the mean and standard deviation of the image channels.

[0018] Furthermore, the ResNet50 classifier module includes a feature extraction subunit and a classification decision subunit. The feature extraction subunit extracts high-dimensional features of the normalized image patch based on the ResNet50 network architecture with frozen convolutional layers.

[0019] The classification decision subunit includes a fully connected layer and a Softmax layer, which maps the high-dimensional features to a space of nine pathological tissue categories, calculates the predicted probability using the Softmax classification probability formula, and determines the tissue classification result based on the predicted probability.

[0020] During the training phase, this module uses the cross-entropy loss function formula to quantify the prediction error and uses the Adam parameter update formula to optimize the parameters of the ResNet50 network architecture.

[0021] Furthermore, the ResNet50 classifier module also includes an interpretability visualization unit.

[0022] This unit utilizes gradient-weighted class activation mapping (GRAM) technology to calculate the gradient information of the target category based on the Grad-CAM weight calculation formula to obtain feature map weights. It then uses a class activation map generation formula to generate a class activation heatmap and uses an image overlay display formula to overlay the class activation heatmap with the original image to display the pathological regions of interest to the ResNet50 classifier module.

[0023] Furthermore, the tissue type-specific feature extraction module includes a multi-scale sampling unit and a specific encoding unit.

[0024] The multi-scale sampling unit acquires the standardized image patch set at different magnifications in parallel. The specific coding unit constructs an independent feature extraction network for each type of pathological tissue, and uses the negative log-partial likelihood loss function formula to guide parameter updates in order to learn feature vectors representing survival risk.

[0025] In addition, the unit performs average pooling on the feature vectors of the same type of pathological tissue from the same patient, and generates the comprehensive feature matrix using the feature matrix construction formula.

[0026] Furthermore, the tissue spatial feature extraction and quantitative analysis module includes a tissue mapping unit and a morphological processing unit.

[0027] The tissue mapping unit traverses the standardized image patch set and reconstructs a multi-tissue type distribution map based on the tissue classification results and spatial coordinates using the tissue binary map mapping formula.

[0028] The morphological processing unit performs a closing operation on the multi-tissue type distribution map, performs connected component analysis using the morphological closing operation formula, and repairs the macroscopic morphological structure of pathological tissues in the multi-tissue type distribution map.

[0029] Furthermore, the organization spatial feature extraction and quantitative analysis module also includes a spatial index calculation unit.

[0030] This unit identifies independent connected regions in the repaired pathological tissue image, calculates the area of ​​the largest connected region of the target pathological tissue type using the maximum connected component area formula, counts the number of lymphocytes inside the tumor region and within a preset distance range extending outward using the tumor intratumoral lymphocyte count formula, calculates the ratio of lymphocytes around the tumor to the lymphocytes inside the tumor using the periodic ratio formula, and calculates the area ratio of pathological tissue to matrix using the tissue-to-matrix area ratio formula.

[0031] This unit uses the area of ​​the largest connected region, the number of lymphocytes, the ratio of lymphocytes around the tumor to lymphocytes inside the tumor, and the area ratio of the pathological tissue to the matrix as quantitative spatial features. It then uses a feature binarization formula to convert the quantitative spatial features into binary features, thereby generating the segmentation feature matrix.

[0032] Furthermore, the multimodal feature fusion and survival prediction module includes a feature fusion unit.

[0033] This unit introduces a weighting mechanism based on the number of pathological tissue blocks to aggregate features at one magnification among the different magnifications. The weight of each type of pathological tissue is calculated using a weighting formula. Then, a multi-scale feature fusion formula is used to fuse the weighted features at one magnification with the features at another magnification among the different magnifications, generating a fused histopathological feature matrix.

[0034] Furthermore, the feature fusion unit is also used to vectorize and splice the fused histopathological feature matrix, the segmentation feature matrix, and the patient's numerical clinical data.

[0035] The patient's numerical clinical data includes the patient's age and coded tumor stage. The vectorization and splicing process follows the complete feature matrix splicing formula to generate the complete multimodal feature matrix.

[0036] Furthermore, the multimodal feature fusion and survival prediction module also includes a model prediction unit.

[0037] The unit receives the complete multimodal feature matrix and uses the survival model to score the patient's risk.

[0038] During the training phase, a hierarchical cross-validation strategy is employed to optimize parameters, and the consistency index formula is used to evaluate model performance. The prediction result of the model with the highest consistency index score is output as the individualized prognostic risk score. The survival models include LASSO Cox regression model, EN Cox regression model, Ridge Cox model, gradient boosting regression tree model, support vector machine model, and random survival forest model.

[0039] A second aspect of the present invention provides a method for prognostic analysis of colorectal cancer that integrates tissue and patch features, applied to the aforementioned prognostic analysis system for colorectal cancer that integrates tissue and patch features. The method includes the following steps: S1. Parse the input whole-slice image of colorectal cancer pathology, select effective regions through a random sampling strategy, generate standardized image patch sets at different magnifications, and convert the standardized image patch sets into Tensor format; S2. Perform feature encoding on the standardized image patch set in Tensor format, and output the predicted probabilities and tissue classification results for nine types of pathological tissues; S3. Extract the standardized image patch set at different magnifications in parallel, and generate a patient-level comprehensive feature matrix using a feature extraction network trained with patient survival data as a supervision signal. S4. Based on the tissue classification results, the standardized image patch set is mapped back to the full slice coordinate space to reconstruct the multi-tissue type distribution map, and morphological repair and connected component analysis are performed on the multi-tissue type distribution map to generate a segmentation feature matrix. S5. Integrate the comprehensive feature matrix, the segmentation feature matrix, and the patient's numerical clinical data to generate a complete multimodal feature matrix, and use the survival model to output an individualized prognostic risk score.

[0040] This invention provides a system and method for prognostic analysis of colorectal cancer that integrates tissue and patch features. It has the following beneficial effects: 1. This invention achieves accurate prediction of survival risk for colorectal cancer patients by constructing a multi-scale, multi-modal deep learning architecture. The system processes pathological images at different magnifications in parallel, taking into account both macroscopic tissue structure and microscopic cell morphology details, avoiding information loss caused by single-scale analysis. At the same time, the system uses a multi-modal feature fusion and survival prediction module to vectorize and stitch together deeply extracted pathological image features, quantified tissue spatial distribution features, and numerical clinical data of patients. This fusion strategy overcomes the limitations of a single data source and can comprehensively characterize the biological heterogeneity and microenvironment state of tumors, improving the accuracy and robustness of risk stratification.

[0041] 2. This invention enhances the interpretability and clinical auxiliary value of pathological analysis by utilizing tissue spatial feature extraction and quantitative analysis modules and visualization technology. The system reconstructs the distribution map of multiple tissue types and performs morphological repair to achieve quantitative characterization of the tumor microenvironment topology. It calculates spatial indicators with clear biological significance, such as the intra-week ratio and the tissue-to-matrix area ratio, and quantifies the specific patterns of immune infiltration and matrix reaction. In addition, the integrated interpretability visualization unit uses gradient-weighted class activation mapping (Grad-CAM) technology to generate heat maps, intuitively displaying the key pathological regions on which the model's decisions are based, verifying the consistency between the algorithm and prior pathological knowledge, and improving the lack of interpretability of deep learning models in medical applications.

[0042] 3. This invention provides a modular and scalable analysis platform architecture with high flexibility and adaptability. The system decouples image processing, classification, feature extraction, and survival prediction into independent modules, supporting the flexible replacement of different backbone networks (such as ResNet, DeepConvSurv) or survival analysis algorithms (such as LASSO Cox, random survival forest) according to clinical needs. This architecture also supports the combined validation of different feature modalities (pathological, spatial, and clinical) through ablation experiments, thereby quantifying the contribution of each feature to the prediction performance. This facilitates the optimization of model configuration for specific datasets or clinical scenarios, providing a general technical tool for the prognostic research and clinical translation of colorectal cancer. Attached Figure Description

[0043] Figure 1 This is a module architecture diagram of the colorectal cancer prognostic analysis system that integrates tissue and patch features according to the present invention; Figure 2 This is a technical architecture diagram of the colorectal cancer pathological image analysis system based on a multi-scale, multi-modal deep learning architecture of the present invention; Figure 3 This is a schematic diagram of the heatmap visualization results of the attention mechanism of the ResNet50 classifier of the present invention; Figure 4This is a schematic diagram showing the visualization results of the spatial organization region features of the present invention.

[0044] The modules include: 100, image block segmentation and standardization module; 200, ResNet50 classifier module; 300, tissue type-specific feature extraction module; 400, tissue spatial feature extraction and quantitative analysis module; and 500, multimodal feature fusion and survival prediction module. Detailed Implementation

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Please see the appendix Figure 1 and 2 The present invention first provides a colorectal cancer prognostic analysis system that integrates tissue and patch features. The system includes: an image patch segmentation and normalization module 100, a ResNet50 classifier module 200, a tissue type-specific feature extraction module 300, a tissue spatial feature extraction and quantitative analysis module 400, and a multimodal feature fusion and survival prediction module 500.

[0047] The image block segmentation and normalization module 100 serves as the system's input processing end, responsible for parsing the original whole slide image (WSI) of colorectal cancer pathology.

[0048] After the full slice image is parsed, the Patch-Extract cutting unit in the module executes a random sampling strategy to select effective regions from the full slice according to a preset ratio (e.g., 5%), and generates standardized image patch sets of uniform size at two magnification ratios of 10x and 40x respectively.

[0049] Subsequently, the Norm-Transform normalization unit introduces a mean-standard deviation normalization algorithm to convert image patches into a Tensor format that meets the input requirements of deep learning models. This step involves the following normalization formula: ; In the formula: For the standardized image tensor; The input is the raw image patch data; This is the mean vector of the image channels; This is the standard deviation vector of the image channels.

[0050] The ResNet50 classifier module 200 is connected to the image patch segmentation and normalization module 100 for automatic identification of the organization type of preprocessed image patches.

[0051] This ResNet50 classifier module 200 utilizes a ResNet50 network pre-trained on large-scale datasets such as NCI-CRC-HE-100k as its backbone. It extracts high-dimensional features from images through a feature extraction subunit and outputs predicted probabilities for nine types of pathological tissues (including glandular ADI, background BACK, debris DEB, lymphocytes LYM, mucus MUC, muscle MUS, normal tissue NORM, stroma STR, and tumor TUM) through a classification decision subunit. The classification probabilities are calculated using the Softmax classification probability formula: ; In the formula: For the input sample to belong to the first Predicted probability of similar organizations; For the network output layer targeting the first Logit score of the class; Iterate through the index for the sum of categories; 9 represents the total number of organization categories as 9.

[0052] The tissue type-specific feature extraction module 300 processes image data at different magnifications (10x and 40x) in parallel. The multi-scale sampling unit within the ResNet50 classifier module 200 acquires image patches at the corresponding magnifications, and the specific encoding unit performs patch-level feature extraction for each tissue type using backbone networks such as DeepConvSurv, ResNet18, ResNet50, or MobileNetV3. During training, patient survival data is used as a supervisory signal to extract deep pathological features that characterize survival risk, and features from similar tissues of the same patient are aggregated to generate a comprehensive feature matrix.

[0053] The tissue spatial feature extraction and quantitative analysis module 400 quantifies the spatial structure of the tumor microenvironment based on the classification results of the ResNet50 classifier module 200. This module performs tissue feature extraction, with the tissue mapping unit mapping discrete image patches back to the full-slice coordinate space to generate background-removed tissue contours and color-coded tissue distribution maps. Different colors represent different tissue types, such as red for tumor TUM, brown for stroma STR, and green for lymphocytes LYM. Subsequently, the morphological processing unit and spatial index calculation unit perform closure repair and connected component analysis on the tissue distribution map to calculate the tumor area, lymphocyte infiltration degree, and the ratio of tissue to stroma, generating a segmentation feature matrix.

[0054] The multimodal feature fusion and survival prediction module 500, as the system's output, is responsible for integrating multi-source information and performing the final survival prediction. The feature fusion unit first fuses 10x and 40x pathological features, and then combines them with numerical clinical data (such as age and stage) output from the Clinical Feature Processing (CFR) section, as well as the aforementioned spatial segmentation features. Data concatenation follows the complete feature matrix concatenation formula: ; In the formula: This is the complete multimodal feature matrix used for the final prediction; This is the fused histopathological feature matrix; To segment the feature matrix; Patient age data; This is the encoded tumor staging data.

[0055] Finally, the model prediction unit uses survival models (LASSO Cox regression model, EN Cox regression model, RidgeCox model, Gradient Boosting Regression Tree (GBRT) model, Support Vector Machine (SSVM) model, and Random Survival Forest (RSF) model) to process the complete multimodal feature matrix, perform patient-level risk assessment, and output survival curves (the vertical axis of the survival curve represents survival rate, and the horizontal axis represents time) including low-risk, intermediate-risk, and high-risk, as well as individualized prognostic risk scores.

[0056] The image patch cutting and normalization module 100 mainly consists of a patch-extract cutting unit and a norm-transform normalization unit, which is used to convert high-resolution full-slice images into structured tensor sequences suitable for deep learning model processing.

[0057] The Patch-Extract cutting unit performs pathological image parsing and multi-scale sampling. This unit reads whole-slice images (WSI) of colorectal cancer pathology images through a digital pathology image interface, parses the image file header information to obtain the original scan magnification (e.g., 20x or 40x) and physical resolution. For scale differences caused by different scanners, the Patch-Extract cutting unit calculates the corresponding scaling factor, uniformly mapping the image to the target analysis scale (in this embodiment, an equivalent resolution of 20x is used as the baseline scale).

[0058] During the sampling process, considering that the whole slice image contains a large number of informationless slide background areas, this unit adopts a random sampling strategy based on tissue region detection.

[0059] In practice, the entire slice is first segmented at low resolution (e.g., using Otsu thresholding or HSV color space filtering) to generate a tissue mask to locate the effective tissue region. Then, a grid is constructed within the effective tissue region, and grid coordinates are randomly extracted according to a preset ratio (the preset ratio is pre-set based on the coverage requirements of the effective tissue region in the full slice image; in this embodiment, it is 5%).

[0060] The system uses the generated coordinate index to crop the corresponding image region from the high-resolution layer and uniformly adjusts its size. The pixels are used to generate a standardized set of image patches. This strategy ensures that the samples are representative of the entire image while reducing the redundancy of subsequent calculations.

[0061] The Norm-Transform normalization unit is connected to the Patch-Extract cutting unit, receiving its output set of normalized image patches. It is responsible for eliminating feature distribution shifts caused by variations in coloring (inconsistent H&E coloring depth) and scanner light source differences. This unit first converts the pixel data of the image patches from unsigned integers to floating-point tensors, and linearly maps the pixel value range from [0,255] to [0,1]. Based on this, the Norm-Transform normalization unit introduces a mean-standard deviation normalization algorithm to numerically correct the RGB channels of the image tensor, ensuring that the feature distribution of the input data conforms to a standard normal distribution, thus adapting to subsequent deep learning models based on ImageNet pre-trained weights. This process involves the following normalization formulas: ; In the formula: This is the standardized image tensor, which is the feature map that has been preprocessed and is ready to be input into the classification network; The input is the raw image patch data (already converted to tensor format); This is the image channel mean vector, used to center the data distribution. In this embodiment, its value is set to [value to be filled in]. ; This is the standard deviation vector of the image channels, used to scale the data dispersion. In this embodiment, its value is set to [value to be filled in]. .

[0062] See attached document Figure 3 The ResNet50 classifier module 200 is connected to the image patch segmentation and normalization module 100, and is responsible for feature encoding and tissue type discrimination of the input normalized image patches. The ResNet50 classifier module 200 mainly consists of a feature extraction subunit, a classification decision subunit, and an interpretability visualization unit.

[0063] The feature extraction subunit is built on a deep residual network (ResNet50) architecture, which consists of multiple residual blocks composed of convolutional layers, batch normalization layers, and activation functions. To leverage general visual features learned on large-scale natural image datasets such as ImageNet, the feature extraction subunit loads pre-trained weights and employs a transfer learning strategy.

[0064] In practice, all convolutional layer parameters of the ResNet50 network except for the fully connected layers at the end are frozen to keep them unchanged during training, and only the high-dimensional abstract features extracted from them are utilized.

[0065] The classification decision subunit is connected after the feature extraction subunit, reconstructing a classification head structure adapted to this task. This structure sequentially includes a linear layer with an input dimension of 2048 and an output dimension of 1024, a Dropout layer with a dropout rate of 0.5 (to prevent overfitting), and a linear classification layer with an output dimension of 9.

[0066] This classification decision subunit maps the extracted features to the nine tissue category spaces required for colorectal cancer pathological analysis, including glands (ADI), background (BACK), debris (DEB), lymphocytes (LYM), mucus (MUC), muscle (MUS), normal tissue (NORM), stroma (STR), and tumor (TUM).

[0067] The Logit score output by the model needs to be processed by a normalized exponential function to convert it into a probabilistic form. The classification decision unit executes the Softmax classification probability formula: ; In the formula: For the input sample to belong to the first Predicted probability of similar organizations; For the network output layer targeting the first The class's logit score is the direct output of the linear layer at the end of the classification decision subunit; 9 represents that the total number of tissue categories is 9, corresponding to the nine types of pathological tissues in this embodiment. The exponentiation operator, specifically representing the exponentiation of natural constants. Exponentiation with base .

[0068] During the model training phase, this system employs a supervised learning paradigm, using the expert-annotated NCT-CRC-HE-100k dataset to iteratively optimize the parameters of the classification decision subunits. The training objective is to minimize the difference between the predicted probability distribution and the true label distribution, a process quantified using the cross-entropy loss function formula: ; In the formula: This is the cross-entropy loss value, used to measure the prediction error under the current model parameters; The training batch size, i.e. the total number of samples, is set to 64 in this embodiment; For the sample Category The true label is encoded using one-hot encoding, meaning that the value is 1 when the sample belongs to the class, and 0 otherwise. For the sample Category The predicted probability; It is the natural logarithm function.

[0069] To efficiently update network parameters to minimize the loss function, this embodiment employs the Adaptive Moment Estimation (Adam) optimization algorithm. This algorithm combines the advantages of the momentum method and the RMSProp algorithm, dynamically adjusting the learning rate of each parameter based on the first and second moment estimates of the gradient. The parameter update process follows the Adam parameter update formula: ; In the formula: For the updated network parameters; These are the network parameters for the current time step; In this embodiment, the initial value is set to the learning rate. ; This is a first-order momentum estimate after bias correction, reflecting the mean direction of the gradient; This is a second-order momentum estimate after bias correction, reflecting the uncentered variance of the gradient; To prevent small constants with denominators of zero, we usually take... Magnitude.

[0070] The training process is performed on a GPU device configured with a CUDA computing environment, and it goes through multiple rounds of iteration until the model converges.

[0071] To verify the biological rationale behind the model's classification decisions and provide visualized diagnostic evidence, the interpretability visualization unit utilizes Gradient Weighted Class Activation Mapping (Grad-CAM) technology to generate class activation maps. This unit first calculates the gradient information of the target class, and then obtains the weights of the feature maps through global average pooling of the gradients. The calculation process involves the Grad-CAM weight calculation formula: ; In the formula: For the first Each feature map is for the category The Grad-CAM weights represent the importance of the feature map for the prediction of the target category; This represents the total number of pixels in the feature map, used for normalization. and Indices for the spatial coordinates of the feature map; The sign for partial derivatives; For the model, target the category The output score (before the Softmax operation); The output of the last convolutional layer Each feature map in coordinates The activation value at that location.

[0072] After obtaining the weights, the interpretability visualization unit performs a weighted summation of the feature maps from each channel and filters out negative activation values ​​to generate a coarse localization map. This process follows the class activation map generation formula: ; In the formula: For categories The class activation heatmap reflects the regions in the image that contribute the most to the classification results. It is a linear rectified function used to retain features that contribute positively to the category and suppress negative effects; For the first Each feature map; The corresponding weights.

[0073] Finally, to visually demonstrate the pathological regions of interest to the model, this unit upsamples the class activation heatmap to the original image size and overlays it onto the original image. This process involves the image overlay display formula: ; In the formula: This is the resulting visual image after overlay, i.e., the result of heatmap overlay; This is the transparency factor, used to adjust the blending ratio between the heatmap and the original image; it is usually set to 0.5. The original input image; For normalization, the heatmap values ​​are mapped to the [0,1] interval for easier visualization rendering; Activate the heatmap for this class.

[0074] Through the visualization process described above, for images identified as normal (Original image: normal), the red highlighted areas in the corresponding heatmap (Normal probability: 100%) accurately cover normal glandular structures; for images identified as tumors (Original image: Tumor), the red highlighted areas in the corresponding heatmap (Tumor probability: 99.98%) precisely focus on nests of atypically proliferating tumor cells, ignoring non-specific stromal background. This visualization result validates that the classifier can make decisions based on pathologically significant morphological features, rather than relying on background noise.

[0075] The tissue type-specific feature extraction module 300 includes a multi-scale sampling unit and a specific coding unit, which aims to mine the prognostic association features hidden in each tissue type at different magnifications.

[0076] The multi-scale sampling unit is based on the multi-resolution pyramid structure of the whole slice image and extracts image patches at 10× magnification and 40× magnification in parallel.

[0077] The 10× data primarily preserves the macroscopic arrangement and interstitial background information of the tissue, while the 40× data focuses on microscopic pathological details such as nuclear morphology, chromatin texture, and mitotic figures. This dual-scale sampling strategy ensures that feature extraction covers the complete morphological spectrum from the cellular to the tissue level.

[0078] For each of the nine tissue types (glandular ADI, background BACK, debris DEB, lymphocyte LYM, mucus MUC, muscle MUS, normal tissue NORM, stroma STR, and tumor TUM) output by the ResNet50 classifier module 200, a separate feature extraction network is constructed and trained.

[0079] Unlike traditional general feature extraction, this unit maintains a dedicated deep learning model (e.g., DeepConvSurv model) for each type of organization, which means that the system contains a total of nine parallel sub-networks.

[0080] During the training phase, these subnetworks do not use category labels as supervision signals, but instead directly utilize patients' clinical survival data (survival time and survival status) for supervised training.

[0081] This mechanism allows each subnetwork to focus on learning morphological variations relevant to patient prognosis in that specific tissue type. For example, the tumor subnetwork will focus on identifying atypia and invasive characteristics of tumor cells, while the lymphoid subnetwork will focus on identifying patterns of immune cell infiltration.

[0082] To establish a mathematical correlation between image features and patient survival risk, the specific coding unit employs a negative log-partial likelihood loss function (Cox Loss) as its optimization objective during training. This loss function aims to maximize the probability of observed survival events, ensuring that the model's output risk value accurately reflects the ranking of patient survival times. The optimization process involves the formula for the negative log-partial likelihood loss function: ; In the formula: The survival analysis loss value is used to guide the parameter updates of the feature extraction network; An index of samples where survival events (such as death) have occurred; For the sample index of risk set, i.e. survival time Greater than or equal to the sample Survival time All samples; For the sample The survival event state, which takes a value of 1 when the event occurs and a value of 0 when the event is deleted; Samples predicted by the model The survival risk value is derived from the output of the input image patch after forward propagation through a convolutional neural network. Samples predicted by the model Survival risk value.

[0083] By minimizing this loss function, the model can learn to assign risk values ​​to high-risk patients. Characteristic expression of pushing up the risk value of high-risk and low-risk patients.

[0084] After model training is complete, the specific encoding unit performs inference and feature aggregation operations. For any patient, the system inputs all image patches in their WSI that are classified into a specific tissue type (e.g., tumor TUM) into the corresponding dedicated sub-network to obtain a set of risk feature vectors.

[0085] Subsequently, an average pooling strategy was used to aggregate the vectors in this group to generate a representative feature vector of the patient in this tissue type (e.g., This process is performed sequentially on nine types of tissues, ultimately concatenating the feature vectors of all types to form a patient-level pathological feature description. This description is defined using a feature matrix construction formula: ; In the formula: The patient-level pathological feature matrix is ​​a comprehensive vector that integrates prognostic information from all tissue components in the whole slide. , , , , , , , and These are the arithmetic mean of the feature vectors of all image patches for the corresponding tissue type (gland, background, debris, lymph, mucus, muscle, normal, stroma, tumor) for the patient. This matrix is ​​directly used as one of the inputs to the subsequent multimodal fusion module, providing in-depth pathological morphological basis for survival prediction.

[0086] See attached document Figure 4 The tissue spatial feature extraction and quantitative analysis module 400 is connected to the ResNet50 classifier module 200. Utilizing the image patch category labels generated by classification and their spatial coordinates in the original whole slice, it reconstructs the macroscopic topology of the tissue and quantifies the characteristics of the tumor microenvironment. This tissue spatial feature extraction and quantitative analysis module 400 mainly consists of a tissue mapping unit, a morphological processing unit, and a spatial index calculation unit.

[0087] The tissue mapping unit performs reverse tissue mapping from discrete image patches to a continuous tissue distribution map.

[0088] This unit traverses all image blocks of the whole-slice image and, based on the predicted category output by the ResNet50 classifier (e.g., tumor TUM, stroma STR, lymphocyte LYM, etc.), labels the pixel region corresponding to each image block with a specific grayscale value or color code. After mapping processing, the original whole-slice image (left image) removes the background regions without information (middle image) and generates a color-coded tissue map containing spatial location information (right image), where red areas represent tumor tissue, brown areas represent stroma, and green areas represent lymphocytes.

[0089] To facilitate subsequent mathematical morphological analysis, this unit generates pixel-level binary mask images for each tissue type. This mapping process follows the tissue binary map mapping formula: ; In the formula: Coordinates in the full slice coordinate system Pixel value at; This indicates that the pixel belongs to the target tissue type (foreground) currently being analyzed; This indicates that the pixel belongs to the background or other organizational type.

[0090] Since image patch-based classification may result in discontinuities at the edges of tissue regions or the presence of noise holes inside, the morphological processing unit performs a closing operation on the generated binary mask image.

[0091] Closure operations, consisting of a cascade of dilation and erosion operations, connect adjacent fracture regions and fill tiny pores, thereby smoothing tissue boundaries and restoring biologically significant macroscopic morphological structures. This process involves the morphological closure operation formula: ; In the formula: This is a binary image after morphological restoration; The initial binary image is the input. As a predefined morphological structural element, this embodiment adopts A rectangular kernel for pixels; This is a morphological dilation operator used to expand the foreground boundary; This is a morphological erosion operator used to shrink the foreground boundary.

[0092] The spatial index calculation unit extracts quantitative spatial features with prognostic value based on the repaired tissue image through a connected component analysis algorithm.

[0093] This unit first identifies all independent connected components in the binary image and calculates their areas, then selects the largest connected component to characterize the dominant load or distribution range of that tissue type across the entire image. The calculation of the largest connected component area follows the formula for the largest connected component area: ; In the formula: The maximum connected region area of ​​the target organization type (measured in pixels or physical area). The total number of connected components identified in the image; For the first The area of ​​each connected component; To calculate the maximum value.

[0094] To quantify the immune infiltration status within the tumor microenvironment, the spatial index calculation unit calculates the number of lymphocytes within the tumor region. This calculation defines the spatial extent of the tumor region and counts the lymphocyte pixels falling within that range. The calculation process involves the following formula for calculating the number of lymphocytes within the tumor: ; In the formula: The number of lymphocytes within the tumor region and its peripheral extension; The pixel coordinate index of the full slice image; Binary mask for tumor region The value at that location (0 or 1); For the binary mask of the lymphocyte region in The value at; This is a spatial expansion window function used to expand each foreground pixel of the tumor region outward by a preset distance (the preset distance is pre-set based on the physical range of the tumor edge microenvironment interaction area, for example, 20 pixels) to cover the tumor edge microenvironment interaction area.

[0095] Furthermore, to assess the spatial distribution patterns of immune cells (such as immune rejection or immune inflammation), this unit calculates the ratio of peritumoral lymphocytes to intratumoral lymphocytes. This indicator reflects whether lymphocytes have successfully penetrated the stromal barrier and entered the tumor core. The calculation follows the intra-weekly ratio formula: ; In the formula: This is the ratio within the week; a lower value generally indicates stronger immune infiltration. The number of lymphocytes in the area surrounding the tumor (i.e., the ring-shaped area within a specific distance outside the tumor boundary); This refers to the number of lymphocytes inside the tumor calculated above; It is a smoothing coefficient used to prevent numerical instability when the denominator is zero.

[0096] Furthermore, this unit calculates the area ratio of specific tissues (such as tumors, mucus, or debris) to the stroma to quantify the impact of matrix response on tumor growth. This calculation involves the formula for the tissue-to-stromal area ratio: ; In the formula: The area ratio of a specific type of tissue to the matrix; This represents the maximum area of ​​connected components for this type of organization. This represents the total area of ​​the matrix tissue in the whole section.

[0097] Finally, to eliminate the influence of outliers and adapt to certain survival analysis models sensitive to discrete features, the spatial index calculation unit converts the aforementioned continuous quantitative spatial features into binary features. This process is based on the statistical distribution of the dataset, using the median as a threshold for segmentation. The conversion process follows the feature binarization formula: ; In the formula: These are the binarized eigenvalues, used to construct the final segmentation feature matrix. ; The original quantitative spatial characteristic values ​​(e.g.) , wait); This is the median of the feature across the entire training dataset.

[0098] The multimodal feature fusion and survival prediction module 500 is connected to the tissue type-specific feature extraction module 300 and the tissue spatial feature extraction and quantitative analysis module 400, and is responsible for integrating heterogeneous multi-source data streams into a unified prognostic risk characterization. This multimodal feature fusion and survival prediction module 500 mainly includes a feature fusion unit and a model prediction unit.

[0099] The feature fusion unit first processes pathological image features from different magnifications. Considering that under low magnification (10×), the area ratio of different tissue types (i.e., the number of image blocks) directly reflects the growth pattern and microenvironment composition of tumor tissue, this unit introduces a weighting mechanism based on the number of tissue blocks to aggregate 10× features.

[0100] For a specific patient, the system calculates weights based on the relative abundance of various tissue image patches to enhance the expression of dominant tissue features while preserving the contribution of sparse key tissues. The weights are calculated using a weighted weighting formula: ; In the formula: For the first The patient was targeted at the first Normalized weights for class-based organizations; For the first The patient was classified as number one. Number of image patches in the organization; This represents the total number of organization categories.

[0101] After obtaining the weighted 10× features, the feature fusion unit fuses them with the 40× features that focus on microscopic cellular details.

[0102] This embodiment employs an arithmetic mean fusion strategy to balance the contributions of macroscopic tissue structure information and microscopic cell morphology information to prognosis, thereby generating a fused histopathological feature matrix. This process involves multi-scale feature fusion formulas: ; In the formula: This is the fused histopathological feature matrix, which integrates multi-scale pathological visual information; Based on the aforementioned weights The weighted feature matrix obtained under a 10x lens; This is the feature matrix obtained through mean pooling under a 40x magnification lens.

[0103] To achieve complementarity between clinical prior knowledge and deep learning features, the feature fusion unit further vectorizes and concatenates the fused pathological features, spatial segmentation features, and patient clinical baseline data. Clinical data includes patient age (numerical) and tumor stage (ordinal encoding, e.g., Stage I to IV are encoded as 1 to 4 respectively).

[0104] Through this splicing operation, the system constructs a high-dimensional descriptor that includes biological morphology, spatial microenvironment structure, and clinical status. This splicing process follows the complete feature matrix splicing formula: In the formula: This is the complete multimodal feature matrix used as input to the survival prediction model; This refers to the fusion pathological features obtained from the aforementioned calculations; The binarized segmentation feature matrix is ​​output by the organization spatial feature extraction and quantitative analysis module. Normalized patient age data; This is the encoded tumor staging data.

[0105] The model prediction unit receives the complete multimodal feature matrix and uses machine learning survival analysis algorithms to score the patient's risk.

[0106] In this embodiment, the unit integrates multiple survival models, which are analyzed using machine learning survival algorithms, including LASSO Cox regression model, EN Cox regression model, Ridge Cox model, Gradient Boosting Regression Tree Model (GBRT), Support Vector Machine Model (SSVM), and Random Survival Forest Model (RSF).

[0107] During the training phase, a five-fold stratified cross-validation strategy is employed to optimize model parameters using the training set data, establishing a non-linear mapping between feature vectors and survival times. To evaluate and select the best prediction model, this unit calculates the consistency index (C-index), which measures the degree of consistency between the model's predicted hazard ranking and survival time ranking. Performance evaluation involves the consistency index (C-index) formula: ; In the formula: This is the consistency index, ranging from 0.5 to 1.0. A higher value indicates higher prediction accuracy. The number of all comparable sample pairs in the test set (i.e., at least one of the samples has experienced a survival event). Patients and patients Survival time; The model outputs for patients and patients The predicted prognostic risk value; This is an indicator function. The function value is 1 when the condition in parentheses is true (i.e., patients with shorter survival times have higher predicted risk values ​​and the sorting is correct), and 0 otherwise.

[0108] Ultimately, the system outputs the model prediction result with the highest C-index score, which serves as the patient's individualized prognostic risk score, providing auxiliary reference for clinical treatment decisions.

[0109] The system locates the effective tissue region based on a tissue region detection algorithm (such as Otsu threshold segmentation). Subsequently, the system executes a random sampling strategy, generating sampling coordinates within the effective region according to a preset sampling ratio (e.g., 5%), and cropping uniformly sized regions at both 10x (macro scale) and 40x (micro scale) magnification. Image blocks (pixels) are used to construct a standardized image block set.

[0110] To adapt to the input requirements of deep learning models, the system further performs standardization processing on the image patch set, converting pixel data into floating-point tensors, and using standardization formulas to perform numerical correction of the mean and standard deviation of the RGB channels to eliminate color and illumination differences.

[0111] S2. Perform feature encoding on the standardized image patch set in Tensor format, and output the predicted probabilities and tissue classification results for nine types of pathological tissues.

[0112] In this step, the preprocessed tensor data is fed into the ResNet50 classifier module. The system uses a ResNet50 backbone network with frozen convolutional layers to extract high-dimensional visual features of image patches, and then maps the features to a nine-dimensional category space (covering glands, background, debris, lymph, mucus, muscle, normal, stroma, and tumor) through a classification decision subunit containing fully connected layers.

[0113] The system uses the Softmax classification probability formula to calculate the probability value of each image patch belonging to each pathological category, and selects the category with the highest probability as the tissue classification result for that image patch. During the model construction phase, this classification process is optimized using the cross-entropy loss function formula and the Adam parameter update formula. Furthermore, heatmaps can be generated using the Grad-CAM weight calculation formula and the class activation map generation formula to verify the rationality of the classification focus areas.

[0114] S3. Extract the standardized image patch set at different magnifications in parallel, and generate a patient-level comprehensive feature matrix using a feature extraction network trained with patient survival data as a supervision signal.

[0115] In this step, the system processes image patch data at 10x and 40x resolutions in parallel. For each type of pathological tissue identified by the ResNet50 classifier, the system calls the corresponding independent feature extraction network (such as DeepConvSurv) to perform deep feature encoding.

[0116] These networks utilize the negative log-partial likelihood loss function (Cox Loss) during training to directly correlate patient survival time and status, thereby capturing morphological features closely related to prognosis. The system performs average pooling on feature vectors of similar pathological tissues from the same patient and concatenates the features from nine tissue types, using a feature matrix construction formula to generate a patient-level comprehensive feature matrix containing multi-scale information.

[0117] S4. Based on the tissue classification results, the standardized image patch set is mapped back to the full slice coordinate space to reconstruct the multi-tissue type distribution map, and morphological repair and connected component analysis are performed on the multi-tissue type distribution map to generate a segmentation feature matrix.

[0118] In this step, the system uses the tissue binary map mapping formula to reconstruct a pixel-level multi-tissue type distribution map at the full slice scale based on the spatial coordinates and classification labels of each image patch.

[0119] To eliminate the boundary discontinuities caused by block processing, the system uses morphological closing operation formulas to repair the distribution map, smoothing tissue boundaries and filling holes.

[0120] Based on this, the system performs quantitative spatial analysis: the distribution range of major tissues is calculated using the formula for the area of ​​the largest connected component; the immune infiltration pattern is quantified using the formulas for calculating the number of lymphocytes in the tumor and the weekly ratio; and the degree of matrix reaction is assessed using the formula for the ratio of tissue to matrix area.

[0121] The calculated quantitative indicators are processed by the feature binarization formula and then combined to generate a segmentation feature matrix describing the spatial structure of the tumor microenvironment.

[0122] S5. Integrate the comprehensive feature matrix, the segmentation feature matrix, and the patient's numerical clinical data to generate a complete multimodal feature matrix, and use the survival model to output an individualized prognostic risk score.

[0123] In this step, the system first integrates multi-scale features in the feature fusion stage, uses a weighted weight calculation formula to weight the 10x features according to the number of tissue blocks, and uses a multi-scale feature fusion formula to combine the 40x features.

[0124] Subsequently, the system vectorizes and concatenates the fused histopathological features, the segmentation feature matrix generated in step S4, and the coded patient clinical data (age, stage). This concatenation process follows the complete feature matrix concatenation formula to construct a high-dimensional complete multimodal feature matrix.

[0125] Finally, the system inputs the matrix into a pre-trained survival model, and calculates the patient's risk value through model inference.

[0126] Survival models include LASSO Cox regression model, EN Cox regression model, Ridge Cox model, Gradient Boosting Regression Tree (GBRT) model, Support Vector Machine (SSVM) model, and Random Survival Forest (RSF) model.

[0127] The model has undergone performance evaluation and optimization during the training phase using the C-index formula, and ultimately outputs an individualized prognostic risk score with the highest accuracy, providing a stratification basis for clinical practice.

[0128] The colorectal cancer prognostic analysis system and method that integrates tissue and patch features provided in this invention integrates multi-scale pathological visual features, tissue spatial structure quantitative indicators and clinical baseline data to construct a colorectal cancer prognostic assessment framework that includes macroscopic structure and microscopic details.

[0129] Unlike conventional methods that rely solely on local image patch texture features or single-scale analysis, this system utilizes a tissue spatial feature extraction and quantitative analysis module to achieve digital reconstruction of the tumor microenvironment topology.

[0130] By quantitatively calculating the spatial interaction between tumors and immune cells (such as the intra-week ratio) and the area ratio of different tissue components, the system can capture structured prognostic factors that are difficult to extract from traditional deep learning black-box models, thereby effectively characterizing the tumor's immune escape mechanism and matrix response status.

[0131] At the multimodal fusion level, the system employs a multi-scale sampling and weighted fusion strategy, taking into account both tissue structure information under low magnification and cell nuclear morphology details under high magnification, reducing the loss of cross-scale information. Furthermore, by incorporating clinical data such as patient age and stage, the survival prediction model is not only based on pixel information from pathological images but also integrates the patient's overall clinical condition, thereby improving the accuracy of risk stratification.

[0132] In addition, the interpretability visualization function integrated into the ResNet50 classifier module can intuitively display the pathological regions on which the model makes decisions, verifying the consistency between the algorithm and prior pathological knowledge.

[0133] In summary, this system not only provides a high-precision survival risk prediction tool, but also offers interpretable support for clinicians to understand tumor heterogeneity and develop individualized treatment plans through quantitative spatial indicators and visual heat maps.

Claims

1. A colorectal cancer prognostic analysis system integrating tissue and patch features, characterized in that, include: The image patch segmentation and normalization module is used to parse the input whole slice image of colorectal cancer pathology, select effective regions from the whole slice image of colorectal cancer pathology through a random sampling strategy, generate normalized image patch sets at different magnifications, and convert the normalized image patch sets into Tensor format. The ResNet50 classifier module is used to encode features of the normalized image patch set in Tensor format and output the predicted probabilities and tissue classification results for nine types of pathological tissues. The tissue type-specific feature extraction module is used to extract the standardized image patch set at different magnifications in parallel, and to generate a patient-level comprehensive feature matrix using a feature extraction network trained with patient survival data as a supervision signal. The tissue spatial feature extraction and quantitative analysis module is used to map the standardized image patch set back to the full slice coordinate space based on the tissue classification results to reconstruct the multi-tissue type distribution map, and to perform morphological repair and connected component analysis on the multi-tissue type distribution map to generate a segmentation feature matrix. The multimodal feature fusion and survival prediction module is used to fuse the comprehensive feature matrix, the segmentation feature matrix, and the patient's numerical clinical data to generate a complete multimodal feature matrix, and to output an individualized prognostic risk score using a survival model.

2. The colorectal cancer prognostic analysis system integrating tissue and patch features according to claim 1, characterized in that, The image block segmentation and normalization module includes: The Patch-Extract cutting unit is used to parse the head information of the whole slide image of colorectal cancer pathology to obtain the original magnification, calculate the scaling factor, and use a random sampling strategy based on tissue region detection to screen effective tissue regions from the whole slide image of colorectal cancer pathology according to a preset ratio and extract the standardized image patch set, wherein the different magnifications include at least 10x magnification and 40x magnification. The Norm-Transform normalization unit is used to receive the normalized image patch set and convert it into Tensor format. Specifically, it converts the pixel data of the normalized image patch set into floating-point tensors and uses normalization operation formulas to correct the mean and standard deviation of the image channels. The preset ratio is set in advance based on the coverage requirements of the effective tissue area in the whole slice image of colorectal cancer pathology.

3. The colorectal cancer prognostic analysis system integrating tissue and patch features according to claim 1, characterized in that, The ResNet50 classifier module includes: The feature extraction subunit, based on a ResNet50 network architecture with frozen convolutional layers, is used to extract high-dimensional features from the normalized image patch. The classification decision subunit includes a fully connected layer and a Softmax layer, which is used to map the high-dimensional features to a space of nine pathological tissue categories, calculate the predicted probability using the Softmax classification probability formula, and determine the tissue classification result based on the predicted probability. The ResNet50 classifier module quantifies the prediction error using the cross-entropy loss function during the training phase and optimizes the parameters of the ResNet50 network architecture using the Adam parameter update formula.

4. The colorectal cancer prognostic analysis system integrating tissue and patch features according to claim 3, characterized in that, The ResNet50 classifier module also includes an interpretable visualization unit; The interpretable visualization unit is used to calculate the gradient information of the target category based on the Grad-CAM weight calculation formula to obtain the feature map weight using gradient weighted class activation mapping technology, generate a class activation heatmap using the class activation map generation formula, and overlay the class activation heatmap with the original image using the image overlay display formula to display the pathological regions of interest of the ResNet50 classifier module.

5. The colorectal cancer prognostic analysis system integrating tissue and patch features according to claim 1, characterized in that, The tissue type-specific feature extraction module includes: A multi-scale sampling unit is used to acquire the standardized image patch set at different magnifications in parallel; Specific coding units are used to construct independent feature extraction networks for each type of pathological tissue, and use the negative log-partial likelihood loss function formula to guide parameter updates in order to learn feature vectors that represent survival risk. The specific coding unit is also used to perform average pooling on the feature vectors of the same type of pathological tissue from the same patient, and to generate the comprehensive feature matrix using the feature matrix construction formula.

6. The colorectal cancer prognostic analysis system integrating tissue and patch features according to claim 1, characterized in that, The tissue spatial feature extraction and quantitative analysis module includes: The tissue mapping unit is used to traverse the standardized image patch set and reconstruct a multi-tissue type distribution map based on the tissue classification results and spatial coordinates using the tissue binary map mapping formula. The morphological processing unit is used to perform a closing operation on the multi-tissue type distribution map, perform connected component analysis using the morphological closing operation formula, and repair the macroscopic morphological structure of the pathological tissues in the multi-tissue type distribution map.

7. The colorectal cancer prognostic analysis system integrating tissue and patch features according to claim 1, characterized in that, The tissue spatial feature extraction and quantitative analysis module also includes a spatial index calculation unit; The spatial index calculation unit is used to identify independent connected regions in the repaired pathological tissue image and calculate the maximum connected region area of ​​the target pathological tissue type using the maximum connected block area formula. The formula for calculating the number of lymphocytes within a tumor is used to count the number of lymphocytes within the tumor region and within a predetermined distance extending outward. The ratio of lymphocytes surrounding the tumor to lymphocytes within the tumor was calculated using the intra-week ratio calculation formula. The area ratio of pathological tissue to matrix is ​​calculated using the formula for the area ratio of tissue to matrix. The area of ​​the largest connected region, the number of lymphocytes, the ratio of lymphocytes around the tumor to lymphocytes inside the tumor, and the area ratio of the pathological tissue to the matrix are used as quantitative spatial features. The quantitative spatial features are converted into binary features using a feature binarization formula to generate the segmentation feature matrix. The preset distance is predetermined based on the physical range of the tumor edge microenvironment interaction zone.

8. The colorectal cancer prognostic analysis system integrating tissue and patch features according to claim 1, characterized in that, The multimodal feature fusion and survival prediction module includes a feature fusion unit; The feature fusion unit is used to introduce a weighted mechanism based on the number of pathological tissue blocks to aggregate features at one of the different magnifications, and to calculate the weight of each type of pathological tissue using a weighted weight calculation formula. The multi-scale feature fusion formula is used to fuse the weighted features at one magnification with the features at another magnification among the different magnifications, and a fused histopathological feature matrix is ​​generated. It is used to vectorize and stitch together the fused histopathological feature matrix, the segmentation feature matrix, and the patient's numerical clinical data; The patient's numerical clinical data includes the patient's age and coded tumor stage. The vectorization and splicing process follows the complete feature matrix splicing formula to generate the complete multimodal feature matrix.

9. The colorectal cancer prognostic analysis system integrating tissue and patch features according to claim 1, characterized in that, The multimodal feature fusion and survival prediction module also includes a model prediction unit; The model prediction unit is used to receive the complete multimodal feature matrix, score the patient risk using the survival model, optimize parameters using a hierarchical cross-validation strategy during the training phase, evaluate model performance using the consistency index formula, and output the model prediction result with the highest consistency index score as the individualized prognostic risk score. The survival models include LASSO Cox regression model, EN Cox regression model, Ridge Cox model, gradient boosting regression tree model, support vector machine model, and random survival forest model.

10. A method for prognostic analysis of colorectal cancer by integrating tissue and patch features, characterized in that, The colorectal cancer prognostic analysis system based on the fusion of tissue and patch features as described in any one of claims 1-9 includes the following steps: S1. Parse the input whole-slice image of colorectal cancer pathology, select effective regions through a random sampling strategy, generate standardized image patch sets at different magnifications, and convert the standardized image patch sets into Tensor format; S2. Perform feature encoding on the standardized image patch set in Tensor format, and output the predicted probabilities and tissue classification results for nine types of pathological tissues; S3. Extract the standardized image patch set at different magnifications in parallel, and generate a patient-level comprehensive feature matrix using a feature extraction network trained with patient survival data as a supervision signal. S4. Based on the tissue classification results, the standardized image patch set is mapped back to the full slice coordinate space to reconstruct the multi-tissue type distribution map, and morphological repair and connected component analysis are performed on the multi-tissue type distribution map to generate a segmentation feature matrix. S5. Integrate the comprehensive feature matrix, the segmentation feature matrix, and the patient's numerical clinical data to generate a complete multimodal feature matrix, and use the survival model to output an individualized prognostic risk score.

Images