A visual analysis prediction method for mutation state of colon cancer ZNF469 gene

Abstract

This invention discloses a visual analysis and prediction method for the ZNF469 gene mutation status in colorectal cancer, belonging to the field of intelligent medical image analysis technology. It involves acquiring whole-slice images of H&E-stained tissue from colorectal cancer patients and corresponding ZNF469 gene mutation status data, constructing an image label pairing dataset; preprocessing the whole-slice images, cutting them into image patches, and using a pre-trained three-class classification model to filter out cancerous region image patches; extracting the macroscopic structural features of each image patch, and simultaneously constructing a nuclear perception map (Transformer) to extract microscopic nuclear morphological features and cell nuclear spatial topological features. This invention achieves rapid prediction of the ZNF469 gene mutation status in colorectal cancer based on pathological images, without relying on gene sequencing technology, reducing detection costs, shortening the diagnostic cycle, and enabling simultaneous mutation status prediction during pathological slide reading, thus improving the efficiency of colorectal cancer diagnosis.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical image analysis technology, and in particular to a visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer. Background Technology

[0002] Colorectal cancer is a highly prevalent malignant tumor worldwide, and its occurrence and development are closely related to mutations in key genes. The ZNF469 gene, a key gene exhibiting mutational characteristics in both early and advanced stages of colorectal cancer, has a mutation status closely related to tumor progression and pathological phenotype. Accurate detection of this gene's mutation status is of significant guiding value for personalized diagnosis and treatment of colorectal cancer. Currently, clinical gene mutation detection mainly relies on gene sequencing technology. This technology is costly, complex, and difficult to correlate with morphological features of pathological images, making it impossible to rapidly predict mutation status during pathological diagnosis.

[0003] Pathological images, as the gold standard for colorectal cancer diagnosis, contain rich information on tissue and cell morphology. Specific gene mutations will exhibit characteristic morphological manifestations in pathological images, becoming an important basis for predicting gene mutation status through visual analysis. Existing gene mutation prediction technologies based on pathological images mostly rely on convolutional neural networks to extract single-level image features, making it difficult to simultaneously capture the macroscopic tissue structure features and microscopic cell nuclear morphology and spatial distribution features of pathological images. Furthermore, they lack sufficient mining of specific features related to gene mutations, and the feature fusion process is easily affected by confounding factors such as staining differences and scanning artifacts, resulting in limited prediction accuracy and robustness.

[0004] Meanwhile, existing technologies have poor interpretability, cannot clearly identify key regions related to gene mutations in pathological images, are difficult to gain clinical acceptance from pathologists, and lack uncertainty assessment of prediction results, which limits their application in clinical practice. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a visual analysis and prediction method for the ZNF469 gene mutation status in colorectal cancer. The following technical solution is adopted:

[0006] A visual analysis and prediction method for ZNF469 gene mutation status in colon cancer, comprising the following steps:

[0007] Step 1: Obtain whole-section images of H&E stained tissue pathology tissues from colorectal cancer patients and corresponding ZNF469 gene mutation status data, and construct an image label pairing dataset;

[0008] Step 2: Preprocess the full slice image, cut it into image blocks, and use a pre-trained three-class classification model to filter out the cancer region image blocks;

[0009] Step 3: Extract the macroscopic structural features of each image patch, and at the same time construct the nuclear sensing map Transformer to extract the microscopic nuclear morphological features and the spatial topological features of the cell nucleus. The nuclear sensing map Transformer introduces ZNF469-specific modulation to enhance the expression of mutation-related features.

[0010] Step 4: Use a multimodal feature fusion method to fuse macroscopic structural features, microscopic kernel morphology features, and spatial topological features to obtain a fused feature vector;

[0011] Step 5: Based on the fused feature vector, construct a multi-task learning classifier with ZNF469 mutation status classification as the main task and tumor staging classification and nuclear atypia score regression as auxiliary tasks, and output the mutation probability of each image block.

[0012] Step 6: Based on the mutation probabilities at the image block level, aggregate the ZNF469 mutation state prediction results at the full slice level, and use the interpretability analysis module to generate a heat map of the mutation-related regions.

[0013] Optionally, step 3, in which the nuclear sensing map Transformer extracts microscopic nuclear morphological features and nuclear spatial topological features, includes the following sub-steps:

[0014] Step 31: Use a nucleus instance segmentation network to segment the image patch into nuclei, and obtain the boundary mask and center point coordinates of each nucleus;

[0015] Step 32: Scale each cell nucleus region to a uniform size and input it into the kernel encoder network to obtain the kernel embedding feature vector;

[0016] Step 33: ZNF469-specific modulation is performed on the nuclear embedding feature vector, and the nuclear morphological features related to mutations are weighted and enhanced through the gene modulation module;

[0017] Step 34: Using the cell nucleus as a node, the modulated nuclear embedding features as node features, and the spatial distance and direction between the nucleus centers as edge features, construct a nuclear relationship graph;

[0018] Step 35: The multi-head graph attention mechanism is used to update the node features of the kernel relation graph. The attention coefficient depends on the node features, edge features, macroscopic structural features extracted in step S3, and ZNF469 prior similarity.

[0019] Step 36: Pool the updated node features to obtain image block-level micro-kernel morphology feature vectors and spatial topological feature vectors.

[0020] Optionally, the kernel encoder in step 32 is a convolutional neural network, which scales each cell kernel region to a fixed size before inputting it and outputs a feature vector of fixed dimensions.

[0021] Optionally, the ZNF469-specific modulation formula in step 33 is as follows:

[0022] ;

[0023] in It is a kernel embedding feature vector that has been specifically modulated by ZNF469. The initial characteristics of the nuclear encoder output, These are manually extracted kernel morphology feature vectors. The weight parameter matrix is ​​a learnable matrix. These are learnable bias parameters. It is the sigmoid activation function. This indicates element-wise multiplication. For the splicing operation, and New vectors are formed by concatenating them according to their dimensions.

[0024] Optionally, in step 34, the kernel relation graph is constructed using a K-nearest neighbor graph, and the edge features are obtained by encoding the Euclidean distance and orientation angle of the kernel center through a multilayer perceptron.

[0025] Optionally, the node feature update formula for the multi-head graph attention mechanism in step 35 is:

[0026] ;

[0027] in For nodes updated using the multi-head graph attention mechanism eigenvectors,

[0028] M represents the number of attention heads. For nodes The neighborhood group, For the first The value transformation matrix of the head, For activation function, For the first Nodes in each attention head For nodes Attention coefficient.

[0029] Optionally, the macroscopic structural features described in step 3 are extracted using a VisionTransformer network. The input image patch is segmented into a patch sequence, and a multi-layer Transformer encoder is used to capture global organizational structural features, outputting the feature vector corresponding to the [CLS] token.

[0030] Optionally, the multimodal feature fusion method in step 4 is as follows: apply self-attention enhancement to macroscopic structural features, microscopic kernel morphology features and spatial topological features respectively, then generate interactive features through intermodal cross-attention, and then remove confounding factors through causal intervention to obtain a causal-enhanced fusion feature vector.

[0031] Optionally, the causal intervention involves constructing a causal graph model, treating staining differences and scanning artifacts during the pathological image acquisition process as confounding variables, and using backdoor adjustment or counterfactual reasoning to remove confounding effects from the interactive features.

[0032] Optionally, in step 5, the multi-task learning classifier shares feature representations, and the total loss function is the weighted sum of the main task loss and the auxiliary task loss; the classifier uses Monte Carlo Dropout technology to estimate uncertainty during the prediction stage, and outputs the mean and variance of the prediction probability. When the variance exceeds a preset threshold, the prediction result of the image patch is marked as requiring manual review; the interpretability analysis module in step S6 includes: using the Transformer's attention mechanism to generate an attention heatmap, locating image regions related to mutation prediction, and training a conditional generative adversarial network to generate corresponding pathological image patches with mutation labels as conditions, and analyzing the typical morphological differences between mutant and wild types by comparing the generated images.

[0033] In summary, the present invention has at least one of the following beneficial technical effects:

[0034] This invention provides a visual analysis and prediction method for the ZNF469 gene mutation status in colorectal cancer. It enables rapid prediction of the ZNF469 gene mutation status in colorectal cancer based on pathological images, without relying on gene sequencing technology, reducing detection costs, shortening the diagnostic cycle, and allowing the prediction of mutation status to be completed simultaneously during the pathology slide reading process, thereby improving the efficiency of colorectal cancer diagnosis.

[0035] Simultaneously, macroscopic structural features, microscopic nuclear morphology features, and nuclear spatial topological features of pathological images are extracted to mine visual features related to ZNF469 mutations from multiple dimensions. Combined with ZNF469-specific modulation to enhance the expression of mutation-related features, the pertinence and comprehensiveness of feature characterization are improved, and the accuracy of mutation state prediction is effectively enhanced.

[0036] By employing a multimodal feature fusion method and introducing causal intervention to remove the influence of confounding factors, the interference of external factors such as staining differences and scanning artifacts on the prediction results is reduced, thereby improving the robustness and generalization ability of the model.

[0037] The main task is to classify ZNF469 mutation status. A multi-task learning classifier is constructed by combining tumor staging classification and nuclear atypia score regression. The feature representation is optimized by the supervision information of the auxiliary task, which further improves the prediction accuracy of the main task.

[0038] The introduction of uncertainty estimation and interpretability analysis modules can not only mark low-confidence prediction results for manual review, but also locate key image regions related to mutations and analyze the morphological differences between mutant and wild types, thereby improving the reliability and interpretability of model results and better meeting the actual needs of clinical pathological diagnosis.

[0039] It can be directly detected based on routine H&E stained pathological whole slide images in clinical practice, without the need for special staining or preprocessing. It is compatible with existing clinical pathological diagnosis procedures and has good prospects for clinical translation and application. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating a visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer according to the present invention.

[0041] Figure 2 This is a schematic diagram comparing the image block preprocessing effects of a specific embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram illustrating the cell nucleus instance segmentation effect according to a specific embodiment of the present invention;

[0043] Figure 4 This is a heatmap of attention prediction for ZNF469 mutations, based on a specific embodiment of the present invention. Detailed Implementation

[0044] The present invention will be further described in detail below with reference to the accompanying drawings.

[0045] This invention discloses a visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer.

[0046] Reference Figures 1-4 Example 1: A visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer, comprising the following steps:

[0047] Step 1: Obtain whole-section images of H&E stained tissue pathology tissues from colorectal cancer patients and corresponding ZNF469 gene mutation status data, and construct an image label pairing dataset;

[0048] Step 2: Preprocess the full slice image, cut it into image blocks, and use a pre-trained three-class classification model to filter out the cancer region image blocks;

[0049] Step 3: Extract the macroscopic structural features of each image patch, and at the same time construct the nuclear sensing map Transformer to extract the microscopic nuclear morphological features and the spatial topological features of the cell nucleus. The nuclear sensing map Transformer introduces ZNF469-specific modulation to enhance the expression of mutation-related features.

[0050] Step 4: Use a multimodal feature fusion method to fuse macroscopic structural features, microscopic kernel morphology features, and spatial topological features to obtain a fused feature vector;

[0051] Step 5: Based on the fused feature vector, construct a multi-task learning classifier with ZNF469 mutation status classification as the main task and tumor staging classification and nuclear atypia score regression as auxiliary tasks, and output the mutation probability of each image block.

[0052] Step 6: Based on the mutation probabilities at the image block level, aggregate the ZNF469 mutation state prediction results at the full slice level, and use the interpretability analysis module to generate a heat map of the mutation-related regions.

[0053] By employing the aforementioned technical approach, ZNF469 gene mutations lead to specific morphological and structural changes in colon cancer cells, which can be visualized through H&E staining pathological images. Collecting paired pathological images and ZNF469 gene mutation data provides supervised learning of image features and mutation states for subsequent models, ensuring that the model can learn visual patterns directly related to the mutation and laying a data foundation for subsequent prediction tasks.

[0054] Whole-section pathological images contain irrelevant areas such as normal tissue and background. Cutting them into fixed-size image patches reduces data dimensionality, improves computational efficiency, and focuses on effective regions. The pre-trained three-class classification model (distinguishing between colonic adenoma, early-stage colon cancer, and advanced-stage colon cancer) has been trained on a large number of samples and has the ability to accurately identify cancerous tissue regions. By using this model to filter out interference from normal tissue and non-target lesion areas, it ensures that the image patches input to subsequent modules are all effective data rich in lesion information, improving the targeting of feature extraction.

[0055] Macroscopic structural features reflect the overall morphological layout and structural pattern of an organization. ZNF469 gene mutations may lead to macroscopic changes such as disordered tissue arrangement and abnormal glandular structure. These global organizational structural features can be captured using the VisionTransformer network. Microscopic nuclear morphological features and nuclear spatial topology are directly related to cellular-level variations. ZNF469 gene mutations may cause uneven nuclear size, abnormal morphology, and disordered distribution. The Nuclear Perception Map Transformer can accurately extract these microscopic features through nuclear segmentation, feature encoding, and relational graph construction. Introducing ZNF469-specific modulation can enhance the expression of features related to this gene mutation, suppress interference from irrelevant features, and improve the correlation between features and mutation states.

[0056] Macroscopic structural features, microscopic nuclear morphology features, and spatial topological features characterize the pathological manifestations associated with ZNF469 gene mutations from different dimensions. Single-dimensional features are insufficient to fully reflect the complex morphological changes caused by mutations. By enhancing features within each modality through self-attention and generating interactive features through cross-attention between modalities, complementary information and intrinsic relationships between features of different dimensions can be explored. Causal intervention can eliminate the influence of confounding factors such as staining differences and scanning artifacts on feature fusion, ensuring that the final fused feature vector can truly and stably reflect the core information related to ZNF469 gene mutations.

[0057] There is an intrinsic biological link between the ZNF469 gene mutation status and tumor stage and nuclear atypia. Tumor stage reflects the degree of disease progression, and nuclear atypia directly reflects the degree of cell morphological abnormality; both can provide auxiliary information for mutation status determination. A multi-task learning classifier, through shared feature representations, enables the main task (ZNF469 mutation status classification) to optimize the feature extraction process with the help of supervisory signals from auxiliary tasks, improving the model's ability to identify mutation-related features. Monte Carlo Dropout technology can quantify the uncertainty of prediction results, providing an assessment basis for the reliability of the results and reducing the risk of misjudgment.

[0058] Image block-level mutation probabilities reflect the mutation likelihood of a local area, while the aggregation of probabilities from all image blocks within a whole slice can comprehensively reflect the mutation status of the entire lesion tissue, improving the comprehensiveness and accuracy of prediction results. The interpretability analysis module locates mutation-related regions using attention heatmaps, visually demonstrating the basis of the model's judgments. Simultaneously, by generating typical pathological images of mutant and wild-type variants through generative adversarial networks, it clearly reveals the morphological differences between the two, enhancing the credibility and clinical interpretability of the prediction results, and facilitating verification and reference by pathologists.

[0059] Example 2, step 3 of the nuclear sensing map Transformer extracts microscopic nuclear morphological features and nuclear spatial topological features, including the following sub-steps:

[0060] Step 31: Use a nucleus instance segmentation network to segment the image patch into nuclei, and obtain the boundary mask and center point coordinates of each nucleus;

[0061] Step 32: Scale each cell nucleus region to a uniform size and input it into the kernel encoder network to obtain the kernel embedding feature vector;

[0062] Step 33: ZNF469-specific modulation is performed on the nuclear embedding feature vector, and the nuclear morphological features related to mutations are weighted and enhanced through the gene modulation module;

[0063] Step 34: Using the cell nucleus as a node, the modulated nuclear embedding features as node features, and the spatial distance and direction between the nucleus centers as edge features, construct a nuclear relationship graph;

[0064] Step 35: The multi-head graph attention mechanism is used to update the node features of the kernel relation graph. The attention coefficient depends on the node features, edge features, macroscopic structural features extracted in step S3, and ZNF469 prior similarity.

[0065] Step 36: Pool the updated node features to obtain image block-level micro-kernel morphology feature vectors and spatial topological feature vectors.

[0066] In Example 3, the kernel encoder described in step 32 is a convolutional neural network, which scales each cell kernel region to a fixed size before inputting it and outputs a feature vector of fixed dimensions.

[0067] Example 4, the ZNF469 specific modulation formula in step 33 is:

[0068] ;

[0069] in It is a kernel embedding feature vector that has been specifically modulated by ZNF469. The initial characteristics of the nuclear encoder output, These are manually extracted kernel morphology feature vectors. The weight parameter matrix is ​​a learnable matrix. These are learnable bias parameters. It is the sigmoid activation function. This indicates element-wise multiplication. For the splicing operation, and New vectors are formed by concatenating them according to their dimensions.

[0070] In Example 5, the kernel relationship graph in step 34 is constructed using a K-nearest neighbor graph, and the edge features are obtained by encoding the Euclidean distance and orientation angle of the kernel center through a multilayer perceptron.

[0071] Example 6, the node feature update formula for the multi-head graph attention mechanism in step 35 is:

[0072] ;

[0073] in For nodes updated using the multi-head graph attention mechanism eigenvectors,

[0074] M represents the number of attention heads. For nodes The neighborhood group, For the first The value transformation matrix of the head, For activation function, For the first Nodes in each attention head For nodes Attention coefficient.

[0075] In Example 7, the macroscopic structural features mentioned in step 3 are extracted using a VisionTransformer network. The input image patch is segmented into a patch sequence, and the global organizational structure features are captured by a multi-layer Transformer encoder, outputting the feature vector corresponding to the [CLS] token.

[0076] By employing the above-mentioned technical solutions, the morphology, size, and distribution variations of the cell nucleus are important manifestations of ZNF469 gene mutations at the cellular level. Using a cell nucleus instance segmentation network, each cell nucleus can be accurately separated from its surrounding tissue. Obtaining a boundary mask clearly defines the outline of the cell nucleus, and the coordinates of the center point provide a basis for subsequent analysis of the spatial relationship of the cell nucleus, ensuring that subsequent feature extraction focuses solely on this core target region of the cell nucleus and excludes interference from irrelevant tissues.

[0077] Cell nuclei in different states (mutant and wild-type) differ in morphological details. Scaling each nucleus region to a uniform size can eliminate the impact of size differences on feature extraction, enabling the kernel encoder to extract features stably. The kernel encoder uses a convolutional neural network, which has a powerful ability to capture local features. It can automatically mine subtle morphological features such as texture and staining intensity inside the cell nucleus and transform them into fixed-dimensional kernel embedding feature vectors, thereby achieving a digital representation of cell nucleus morphology.

[0078] The ZNF469 gene mutation leads to specific changes in nuclear morphology, and the initial features output by the nuclear encoder may contain a large number of general features unrelated to the mutation. By using a gene modulation module, the initial nuclear embedding features are concatenated with manually extracted nuclear morphology features. Using a learnable parameter matrix and bias parameters, modulation coefficients are generated through a sigmoid activation function. Element-wise weighting of the initial features enhances the expression of features related to the ZNF469 mutation and suppresses irrelevant features, making the modulated nuclear embedding features more targeted.

[0079] The ZNF469 gene mutation not only affects the morphology of individual cell nuclei but may also alter the spatial arrangement between them. Using cell nuclei as nodes and modulated nuclear embedding features as node features preserves the individual morphological characteristics of each nucleus. Edge features, encoded using a multilayer perceptron based on the Euclidean distance and orientation angle between nuclei, quantify the spatial position and directional relationships between nuclei. Constructing a nuclear relationship graph using a K-nearest neighbor graph focuses on the spatial relationships of the cores, reduces redundant connections, and efficiently represents the spatial topology of the cell nucleus population.

[0080] Spatial associations among cell nuclei are crucial for determining the ZNF469 mutation status, with different types of associations contributing differently to prediction. A multi-head graph attention mechanism captures association information from different dimensions using multiple independent attention heads. The attention coefficient comprehensively considers node features, edge features, macroscopic structural features, and ZNF469 prior similarity, accurately measuring the influence weight of neighboring nodes on the target node. Weighted aggregation after linear transformation of node features using a value transformation matrix updates the node features, making them more reflective of the spatial association patterns between the cell nucleus population and the ZNF469 mutation.

[0081] The features of a single cell nucleus cannot fully reflect the overall features related to the ZNF469 mutation within an image patch. Pooling all updated node features can integrate the morphological and spatial topological features of all cell nuclei within the image patch, forming a unified image patch-level feature vector. This provides a standardized micro-dimensional input for subsequent multimodal feature fusion, ensuring that micro and macro features have a consistent representation form during fusion.

[0082] The VisionTransformer network, by segmenting image patches into a sequence of patches, overcomes the limitations of local feature extraction. Leveraging the self-attention mechanism of a multi-layer Transformer encoder, it captures global dependencies between patches, thereby extracting the overall organizational structure features of the image patches, such as glandular arrangement and tissue texture distribution. These macroscopic structural features are related to tissue-level changes caused by the ZNF469 gene mutation. The feature vector corresponding to the output [CLS] token can condense global organizational structure information, providing comprehensive macroscopic support for subsequent multimodal fusion.

[0083] Example 8, the multimodal feature fusion method in step 4 is as follows: self-attention enhancement is applied to macroscopic structural features, microscopic kernel morphology features and spatial topological features respectively, then interactive features are generated through intermodal cross-attention, and then confounding factors are removed through causal intervention to obtain the causal enhanced fusion feature vector.

[0084] Example 9: The causal intervention constructs a causal graph model, treats staining differences and scanning artifacts in the pathological image acquisition process as confounding variables, and removes confounding effects from interactive features using backdoor adjustment or counterfactual reasoning.

[0085] By adopting the above technical solution, the core principle of multimodal feature fusion is to fully explore the intrinsic correlation and complementary value of macroscopic structural features, microscopic nuclear morphology features and spatial topological features, strengthen effective information and suppress redundant interference through attention mechanism, and eliminate the misleading influence of confounding factors on feature correlation, so as to construct a fusion vector that can truly reflect the biological characteristics related to ZNF469 gene mutation, and provide accurate and robust feature support for subsequent mutation state prediction.

[0086] The principle of self-attention enhancement lies in the fact that there are different degrees of information importance within the features of each modality. Some features are more strongly associated with the ZNF469 gene mutation, while others may be irrelevant and redundant information. By applying the self-attention mechanism to the features of the three modalities respectively, the weight coefficients of each dimension within the feature can be adaptively calculated, highlighting the expression of key features related to mutations, weakening the interference of meaningless information, and making the representation of single-modal features more targeted, thus laying a high-quality foundation for subsequent cross-modal interactions.

[0087] The principle behind cross-modal attention-based generation of interactive features is that macroscopic structural features reflect overall tissue morphological changes, microscopic nuclear morphological features embody morphological variations at the cellular level, and spatial topological features characterize the positional relationships between cell nuclei. These three aspects characterize the pathological changes caused by ZNF469 gene mutations from different dimensions and are intrinsically linked biologically. Through the cross-attention mechanism, the dependencies and synergistic information between different modal features can be captured, such as the correspondence between macroscopic tissue disorder and microscopic nuclear morphological abnormalities, and the correlation between nuclear spatial distribution patterns and tissue structural changes. The generated interactive features can integrate multi-dimensional information to form a more comprehensive mutation-related feature representation, avoiding the limitations of single-modal features.

[0088] The principle of causal intervention to remove confounding factors is that pathological images are easily affected by external factors such as staining differences and scanning artifacts during acquisition. These confounding factors may lead to spurious associations between different modal features, interfering with the model's learning of mutation-related true features. By constructing a causal graph model, the causal relationship between confounding variables and each modal feature and mutation state can be clarified. Using methods such as backdoor adjustment or counterfactual reasoning, the influence of confounding factors on interactive features can be removed, eliminating the interference caused by spurious associations. This ensures that the final fused feature vector can truly reflect the intrinsic causal relationship between the three modal features and the ZNF469 gene mutation, improving the reliability and generalization ability of feature representation.

[0089] In Example 10, in step 5, the multi-task learning classifier shares feature representations, and the total loss function is the weighted sum of the main task loss and the auxiliary task loss. During the prediction phase, the classifier uses Monte Carlo Dropout technology to estimate uncertainty and outputs the mean and variance of the prediction probability. When the variance exceeds a preset threshold, the prediction result of the image patch is marked as requiring manual review. The interpretability analysis module in step S6 includes: using the Transformer's attention mechanism to generate an attention heatmap, locating image regions related to mutation prediction, and training a conditional generative adversarial network to generate corresponding pathological image patches based on mutation labels. By comparing the generated images, the typical morphological differences between mutant and wild types are analyzed.

[0090] By adopting the above technical solution, the core of the shared feature representation in the multi-task learning classifier lies in the inherent biological relationship between the ZNF469 gene mutation state, tumor stage, and nuclear atypia, all of which rely on similar core visual features in pathological images. The shared feature extraction layer can simultaneously capture general features that meet the requirements of both the main and auxiliary tasks, avoiding redundant feature extraction, improving model training efficiency and parameter utilization efficiency, and making the feature representation more generalizable.

[0091] The overall loss function employs a weighted sum of the main task loss and the auxiliary task loss because the auxiliary task provides additional supervisory signals. Tumor staging reflects the degree of disease progression, and nuclear atypia directly reflects the degree of cell morphological abnormalities. The supervisory information from both can constrain the feature extraction process, guiding the model to focus on key features related to tumor malignancy, thereby indirectly improving the accuracy of the main task in classifying ZNF469 mutation states. By appropriately setting weights, the training priorities of different tasks can be balanced, ensuring that the core objective of the main task is not interfered with by the auxiliary task, while fully utilizing the useful information from the auxiliary task.

[0092] The Monte Carlo Dropout technique for uncertainty estimation works by using the Dropout layer multiple times to predict the same image patch during the prediction phase, and then assessing the uncertainty using the statistical properties of the multiple prediction results. The Dropout layer randomly discards some network nodes, simulating the prediction process of different sub-networks. The probability distribution obtained from multiple predictions can be used to calculate the mean and variance. The mean reflects the core tendency of the prediction results, while the variance reflects the stability of the prediction. A larger variance indicates greater uncertainty in the model's judgment of the image patch, which may stem from factors such as blurred image features, rare morphologies, or data noise. In this case, manual verification of the labeling is necessary to reduce the risk of misjudgment and improve the reliability of the prediction results, making it particularly suitable for clinical diagnostic scenarios with stringent accuracy requirements.

[0093] The principle behind generating attention heatmaps using the Transformer's attention mechanism is that when processing image features, the Transformer encoder calculates the association weights of different feature locations through a self-attention mechanism. The weight reflects the contribution of that feature location to the mutation prediction result. Mapping these weights back to the original pathological image generates an attention heatmap, which visually locates key image regions related to ZNF469 mutation prediction. This visualizes the model's decision-making basis, helping pathologists quickly focus on core lesion areas and understand the model's judgment logic.

[0094] The principle behind training a Conditional Generative Adversarial Network (CGN) to generate pathological image patches based on mutation labels is as follows: the CGN comprises a generator and a discriminator. The generator learns to generate pathological image patches that conform to the characteristics of the corresponding mutation label (mutant or wild-type). The discriminator is responsible for distinguishing between generated and real images. Through adversarial training, the generator can progressively generate image patches that are highly similar in morphological features to real pathological images, and these image patches carry typical morphological information of the corresponding mutation label. By comparing the differences between mutant and wild-type generated images, the typical differences in pathological morphology between the two genotypes can be clearly revealed, such as differences in cell nucleus size, arrangement, and staining intensity. This provides an intuitive reference for morphological studies related to ZNF469 gene mutations, while enhancing the interpretability and persuasiveness of the model's prediction results.

[0095] The following specific embodiments illustrate the implementation principle of the present invention:

[0096] Data preparation and dataset construction:

[0097] We collected whole-slice images of H&E-stained histopathological tissues from 300 patients with colorectal cancer. All images were obtained from clinical surgical resection specimens after formalin fixation and paraffin embedding, acquired using a whole-slice scanning system at 20x magnification, with an image resolution of 10000×10000 pixels. Simultaneously, we collected ZNF469 gene mutation status data for each patient, obtained through next-generation sequencing technology, including 120 mutant samples and 180 wild-type samples. We constructed an image-labeled paired dataset by mapping each patient's whole-slice image to its ZNF469 gene mutation status. The training set consisted of 210 samples (84 mutant, 126 wild-type), the validation set of 45 samples (18 mutant, 27 wild-type), and the test set of 45 samples (18 mutant, 27 wild-type). During the partitioning process, we ensured that all data from the same patient were grouped into the same set.

[0098] Whole-slice image preprocessing and cancer region screening:

[0099] The open-source Python tool OpenSlide was used to process the full-slice images, cutting each image into non-overlapping 512×512 pixel image blocks. The percentage of pixels with RGB pixel values ​​below 220 in each image block was calculated, and image blocks with a percentage below 50% were removed to eliminate background noise. Finally, an average of 2000 valid image blocks were obtained for each full-slice image.

[0100] Figure 2 This is a comparison of the preprocessing effects of image patches. The left side shows unfiltered image patches from the original H&E stained whole slice image, which contain a large amount of stroma, adipose tissue, and background noise. The right side shows cancer region image patches after being filtered by a pre-trained three-classification model, which excludes non-cancer tissue and focuses on areas rich in tumor cells, providing high-quality and targeted input data for subsequent multimodal feature extraction.

[0101] The pre-trained three-class classification model uses the InceptionV3 architecture. This model has been trained on 1000 pathological images of colonic adenomas, early-stage colon cancer, and advanced-stage colon cancer, achieving a classification accuracy of 0.92 on the independent test set. All valid image patches are input into this pre-trained model. Image patches that the model classifies as early-stage or advanced-stage colon cancer are selected as cancer region image patches, while image patches classified as colonic adenomas or normal tissue are removed. The training set, validation set, and test set ultimately retain 386,000, 58,000, and 59,000 cancer region image patches, respectively.

[0102] Multi-dimensional feature extraction:

[0103] Macroscopic structural feature extraction:

[0104] A Vision Transformer network was used to extract macroscopic structural features. Each 512×512 pixel image patch was divided into a 16×16 pixel patch sequence, resulting in 1024 patches. Feature extraction was performed using a 12-layer Transformer encoder, with each layer containing 8 attention heads and a hidden layer dimension of 768. The final output is a 768-dimensional feature vector corresponding to the [CLS] token, which serves as the macroscopic structural feature of the image patch.

[0105] Extraction of microscopic nuclear morphology features and spatial topological features of the cell nucleus:

[0106] Cell nucleus instance segmentation: The U-Net++ network was used as the cell nucleus instance segmentation network to segment each cancer region image block, and the boundary mask and center point coordinates of each cell nucleus were obtained. On average, 120 cell nuclei were obtained per image block.

[0107] Figure 3This is a schematic diagram of the cell nucleus instance segmentation effect. Based on the cell nucleus instance segmentation network, each cell nucleus is accurately delineated with a boundary mask (green outline) and the center point coordinates are marked (red dots). This segmentation result lays the foundation for constructing a nuclear-aware graph Transformer, allowing each cell nucleus to serve as a graph node, and its morphological features and spatial location information can be independently encoded and associated.

[0108] Kernel embedding feature vector generation: Each cell nucleus region is scaled to 32×32 pixels and input to a kernel encoder network consisting of 3 convolutional layers, 2 pooling layers and 2 fully connected layers. The convolutional kernel sizes of the convolutional layers are 3×3, 3×3 and 3×3 respectively, and the pooling layers use max pooling. The final output is a 256-dimensional kernel embedding feature vector hi0.

[0109] ZNF469-specific modulation: The morphological feature vector mi of each cell nucleus was manually extracted, including 10 dimensions such as nuclear area, perimeter, roundness, and mean staining intensity. The modulation formula was followed. ; Perform feature modulation, where The learnable weight matrix is ​​256×266. The bias parameters are 256-dimensional learnable parameters, and the modulation coefficients are generated through the sigmoid function. Element-wise weighting is performed to obtain a 256-dimensional modulated kernel embedding feature vector. .

[0110] Kernel Relationship Graph Construction: The K-nearest neighbor algorithm (K=10) is used to construct the kernel relationship graph, with each cell nucleus as a node and the modulated kernel embedding feature vector hi as the node feature. The Euclidean distance and orientation angle between the center point of each cell nucleus and the center points of other cell nuclei are calculated. These two parameters are input into a two-layer multilayer perceptron for encoding to obtain 16-dimensional edge features, thus constructing the complete kernel relationship graph.

[0111] Node feature update: A multi-head graph attention mechanism (M=8 attention heads) is used to update node features, with a weight transformation matrix for each attention head. It is 256×256 dimensional. Attention coefficient. The similarity between node i and node j is calculated by combining feature similarity, edge features, macroscopic structural features, and ZNF469 prior similarity, according to the formula. Update the node features to obtain the updated 256-dimensional node feature vector. .

[0112] Feature pooling: Global average pooling is performed on all updated node feature vectors to obtain a 256-dimensional micro-kernel morphology feature vector; at the same time, the adjacency matrix features of the kernel relationship graph are calculated, including average degree, clustering coefficient, etc., and combined with the pooled node features to generate a 256-dimensional spatial topological feature vector.

[0113] Multimodal feature fusion:

[0114] Self-attention enhancement: Self-attention mechanisms are applied to macroscopic structural features, microscopic kernel morphology features, and spatial topological features respectively. The hidden layer dimension of the self-attention layer of each modality is consistent with the corresponding feature dimension. By calculating the attention weight of each dimension within the feature, the expression of key dimension features is strengthened.

[0115] Intermodal cross-attention: Construct an intermodal cross-attention layer, using macroscopic structural features as query vectors and microscopic kernel morphology features and spatial topology features as key-value vectors, calculate intermodal attention weights, and generate 384-dimensional interactive features; then use microscopic kernel morphology features as query vectors and macroscopic structural features and spatial topology features as key-value vectors to generate another 384-dimensional interactive feature; finally, fuse the interactive features generated in the two steps to obtain an initial 768-dimensional fused feature.

[0116] Causal intervention: A causal graph model was constructed, and staining differences and scanning artifacts were set as confounding variables. The influence coefficient of the confounding variables was calculated by the backdoor adjustment method, and the influence was removed from the initial fusion features. Finally, a 768-dimensional causal-enhanced fusion feature vector was obtained.

[0117] Multi-task learning classifier construction and training:

[0118] The multi-task learning classifier takes the fused feature vector as input and shares the feature extraction layer. The main task is binary classification of ZNF469 mutation status (mutant / wild type), and the auxiliary tasks are binary classification of tumor stage (early / advanced stage) and regression of nuclear atypia score (score range 1-4 points).

[0119] The classifier output layer contains three branches: the main task branch uses the sigmoid activation function and outputs the mutation probability; the tumor staging branch uses the sigmoid activation function and outputs the staging probability; and the nuclear atypia scoring branch uses the linear activation function and outputs the predicted score. The total loss function is L = 0.6L1 + 0.2L2 + 0.2L3, where L1 is the main task cross-entropy loss, L2 is the auxiliary classification task cross-entropy loss, and L3 is the auxiliary regression task mean squared error loss.

[0120] The model was trained using the Adam optimizer with a learning rate of 0.0001, a batch size of 32, and 10,000 training iterations. Performance was evaluated on a validation set every 200 iterations, and the model with the lowest validation set loss was saved. Monte Carlo Dropout was used for prediction, performing 20 predictions for each image patch. The mean and variance of the 20 predictions were calculated, and a variance threshold of 0.05 was set. When the variance exceeded this threshold, the prediction result for that image patch was marked as requiring manual review.

[0121] Full-slice level prediction results and interpretability analysis:

[0122] The average mutation probability of all cancer region image blocks in each whole slice image is taken as the predicted probability of ZNF469 mutation status of that whole slice. The predicted probability threshold is set to 0.5. If the probability is greater than 0.5, it is determined to be mutant; otherwise, it is wild-type.

[0123] In the interpretability analysis module, an attention heatmap is generated based on the Transformer's attention mechanism, mapping the attention weights back to the original image patches. The red areas represent the regions with the highest attention weights, i.e., the key regions related to mutation prediction. Simultaneously, a conditional generative adversarial network is trained. The generator uses a DCGAN architecture, and the discriminator is constructed using three convolutional layers. Using mutation labels as conditions, pathological image patches corresponding to mutant and wild-type mutations are generated respectively. By comparing the generated images, it was found that the mutant image patches exhibit more significant differences in cell nucleus size, more disordered arrangement, and higher staining intensity.

[0124] Figure 4 This is a heatmap of attention for ZNF469 mutation prediction. Using the Transformer's attention mechanism, the model's contribution weights to ZNF469 mutation prediction are mapped back to the original image space to generate the heatmap. Red areas represent high attention, i.e., cell nuclei or tissue structures strongly associated with the ZNF469 mutation identified by the model; blue areas represent low attention. This heatmap visually locates key pathological regions related to the mutation, enhancing the model's interpretability.

[0125] Model performance evaluation:

[0126] On the test set, the method achieved an accuracy of 0.89, a sensitivity of 0.86, a specificity of 0.91, and an AUC of 0.93 for predicting ZNF469 mutation status; an accuracy of 0.87 for tumor staging; and a mean absolute error of 0.32 for regression of nuclear atypia scores. A total of 1200 image blocks requiring manual review were labeled. After review by pathologists, the prediction results of 32 whole slides were corrected, improving the final prediction accuracy to 0.91.

[0127] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer, characterized in that, Includes the following steps: Step 1: Obtain whole-section images of H&E stained tissue pathology tissues from colorectal cancer patients and corresponding ZNF469 gene mutation status data, and construct an image label pairing dataset; Step 2: Preprocess the full slice image, cut it into image blocks, and use a pre-trained three-class classification model to filter out the cancer region image blocks; Step 3: Extract the macroscopic structural features of each image patch, and at the same time construct the nuclear sensing map Transformer to extract the microscopic nuclear morphological features and the spatial topological features of the cell nucleus. The nuclear sensing map Transformer introduces ZNF469-specific modulation to enhance the expression of mutation-related features. Step 4: Use a multimodal feature fusion method to fuse macroscopic structural features, microscopic kernel morphology features, and spatial topological features to obtain a fused feature vector; Step 5: Based on the fused feature vector, construct a multi-task learning classifier with ZNF469 mutation status classification as the main task and tumor staging classification and nuclear atypia score regression as auxiliary tasks, and output the mutation probability of each image block. Step 6: Based on the mutation probabilities at the image block level, aggregate the ZNF469 mutation state prediction results at the full slice level, and use the interpretability analysis module to generate a heat map of the mutation-related regions.

2. The visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer according to claim 1, characterized in that, Step 3, the nuclear sensing map Transformer extracts microscopic nuclear morphological features and nuclear spatial topological features, including the following sub-steps: Step 31: Use a nucleus instance segmentation network to segment the image patch into nuclei, and obtain the boundary mask and center point coordinates of each nucleus; Step 32: Scale each cell nucleus region to a uniform size and input it into the kernel encoder network to obtain the kernel embedding feature vector; Step 33: ZNF469-specific modulation is performed on the nuclear embedding feature vector, and the nuclear morphological features related to mutations are weighted and enhanced through the gene modulation module; Step 34: Using the cell nucleus as a node, the modulated nuclear embedding features as node features, and the spatial distance and direction between the nucleus centers as edge features, construct a nuclear relationship graph; Step 35: The multi-head graph attention mechanism is used to update the node features of the kernel relation graph. The attention coefficient depends on the node features, edge features, macroscopic structural features extracted in step S3, and ZNF469 prior similarity. Step 36: Pool the updated node features to obtain image block-level micro-kernel morphology feature vectors and spatial topological feature vectors.

3. The visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer according to claim 2, characterized in that, The kernel encoder described in step 32 is a convolutional neural network, which scales each cell kernel region to a fixed size before inputting it and outputs a feature vector of fixed dimensions.

4. The visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer according to claim 3, characterized in that, The ZNF469 specific modulation formula mentioned in step 33 is as follows: ； in It is a kernel embedding feature vector that has been specifically modulated by ZNF469. The initial characteristics of the nuclear encoder output, These are manually extracted kernel morphology feature vectors. The weight parameter matrix is ​​a learnable matrix. These are learnable bias parameters. It is the sigmoid activation function. This indicates element-wise multiplication. For the splicing operation, and New vectors are formed by concatenating them according to their dimensions.

5. The visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer according to claim 4, characterized in that, In step 34, the kernel relationship graph is constructed using a K-nearest neighbor graph, and the edge features are obtained by encoding the Euclidean distance and orientation angle of the kernel center through a multilayer perceptron.

6. The visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer according to claim 5, characterized in that, The node feature update formula for the multi-head graph attention mechanism in step 35 is: ； in For nodes updated using the multi-head graph attention mechanism eigenvectors, M represents the number of attention heads. For nodes The neighborhood group, For the first The value transformation matrix of the head, For activation function, For the first Nodes in each attention head For nodes Attention coefficient.

7. The visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer according to claim 6, characterized in that, The macroscopic structural features described in step 3 are extracted using a VisionTransformer network. The input image patch is segmented into a patch sequence, and a multi-layer Transformer encoder captures the global organizational structure features, outputting the feature vector corresponding to the [CLS] token.

8. The visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer according to claim 7, characterized in that, The multimodal feature fusion method in step 4 is as follows: self-attention enhancement is applied to macroscopic structural features, microscopic kernel morphology features and spatial topological features respectively, then interactive features are generated through intermodal cross-attention, and then confounding factors are removed through causal intervention to obtain a causal-enhanced fusion feature vector.

9. The visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer according to claim 8, characterized in that, The causal intervention constructs a causal graph model, treats staining differences and scanning artifacts in the pathological image acquisition process as confounding variables, and removes confounding effects from interactive features using backdoor adjustment or counterfactual reasoning.

10. The visual analysis and prediction method for the ZNF469 gene mutation status in colon cancer according to claim 9, characterized in that, In step 5, the multi-task learning classifier shares feature representations, and the total loss function is the weighted sum of the main task loss and the auxiliary task loss. In the prediction stage, the classifier uses Monte Carlo Dropout technology to estimate uncertainty and outputs the mean and variance of the prediction probability. When the variance exceeds a preset threshold, the prediction result of the image patch is marked as requiring manual review. The interpretability analysis module in step S6 includes: using the Transformer's attention mechanism to generate an attention heatmap, locating image regions related to mutation prediction, and training a conditional generative adversarial network to generate corresponding pathological image patches with mutation labels as conditions. By comparing the generated images, the typical morphological differences between mutant and wild types are analyzed.

Images