Method and device for predicting gene mutation based on WSI image cascade packet level learning

Abstract

The invention discloses a method and equipment for predicting gene mutation based on WSI image cascade package level learning, and the method comprises the steps: cutting a WSI image into color blocks at equal intervals according to the maximum resolution, and obtaining a color block set; performing feature extraction on the color block set of the WSI image by using a fine-tuned ViT model to generate a color block level feature matrix; the color block level feature matrix of one WSI image is regarded as a packet; training a first-level package-level model based on the benign and malignant package-level labels and the color block level features, outputting a first-level WSI package-level feature matrix through the first-level package-level model, and outputting image-level benign and malignant prediction through the first-level package-level model; and training a secondary package level model based on a gene mutation state package level label and the primary WSI package level feature matrix, wherein the secondary package level model is used for identifying the gene mutation state of the tumor region of the WSI image. According to the method, the gene mutation is predicted only through the WSI image through the cascade package level model.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image data information mining, and particularly relates to a method and device for predicting gene mutation based on WSI image cascade package level learning. BACKGROUND

[0002] Gene mutation refers to permanent change of DNA sequence under the action of environment factors, including base substitution, insertion, deletion, duplication, inversion or chromosomal structural variation, etc. Its essence is "spelling error" of genetic information at the molecular level, which can occur in germ cells (heredity) or somatic cells (tumor occurrence). The causes of gene mutation can be divided into endogenous (replication error, spontaneous deamination) or exogenous (ultraviolet, chemical carcinogens, viruses, etc.). It can lead to loss of function / abnormality of protein (such as inactivation of tumor suppressor gene TP53), and even give cells selective advantage (such as EGFR activating mutation driving tumor). As the source of tumor heterogeneity, it can serve as a biomarker for targeted therapy (such as EGFR inhibitor) and immunotherapy (such as TMB evaluation).

[0003] In the era of precision oncology, gene mutation has become a core indicator for evaluating patient survival prognosis and developing individualized treatment. Current gene mutation detection methods mainly include polymerase chain reaction (PCR), next-generation sequencing (NGS) and real-time fluorescence quantitative PCR (RT-qPCR). The detection cost varies from several thousand yuan to tens of thousands of yuan, and the specific price varies with the detection project and detection institution, and the detection effect and accuracy are easily affected by experimental site, reagent, operation, etc.

[0004] At present, a large number of studies have shown the correlation between tumor cell morphology and genes. In the same cancer, gene mutation can significantly change the morphology of tumor cells, for example: it can promote the transformation of cells from close arrangement to loose and extended interstitial morphology, present irregular spindle or star shape, form sheet-like pseudopodia and filamentous pseudopodia, increase cell volume, form membrane protrusions and vesicles, irregular nuclear membrane and even multinucleated nucleus, and reduce heterochromatin in the nucleus and increase nucleolus, thereby making it acquire more invasive and adaptive characteristics. It is found that the presence of certain gene mutation has subtle but repeatable texture, karyotype and gland structure differences on HE sections compared with cells without mutation. Based on this idea, these differences can be quantified by package level learning, that is, a 1-gigapixel WSI image is cut into tens of thousands of color blocks. As long as a few key areas contain mutation signals, the whole slice is considered "mutation positive", which greatly reduces the conventional annotation time. SUMMARY

[0005] The application aims to provide a method and device for predicting gene mutation based on WSI image cascade package level learning, which combines a pathological image deep learning feature extraction model with a cascade package level learning method to realize the prediction of gene mutation only through WSI images, and then realize rapid intraoperative pathological diagnosis, rapidly screen out patients with gene mutation, and improve the efficiency and success rate of clinical trials.

[0006] In a first aspect, the application provides a method for predicting gene mutation based on WSI image cascade package level learning, comprising: Obtaining any paraffin HE stained section of a disease and scanning it into a WSI image; Cutting the WSI image into color blocks according to the maximum resolution, removing invalid color blocks, and obtaining a color block set; Using a ViT model (Vision Transformer) fine-tuned based on a DINOv2 self-supervised learning method to extract features from the color block set of the WSI image, and generating a color block level feature matrix; regarding the color block level feature matrix of a WSI image as a package; Assigning benign and malignant as package level labels to each package, training a first package level model based on the benign and malignant package level labels and the color block level features, obtaining a first color block attention weight through the first package level model, multiplying the first color block attention weight with a first color block descriptor matrix of the WSI image point by point, and outputting a first WSI package level feature matrix, and outputting an image level benign and malignant prediction by the first package level model; Assigning a gene mutation state as a package level label to each package, and training a second package level model based on the gene mutation state package level label and the first WSI package level feature matrix, the second package level model being used to identify the gene mutation state of the tumor region of the WSI image.

[0007] Further preferably, the feature extraction process is as follows: after color normalization of N color blocks of a WSI image, the color block size is unified, and then the ViT model fine-tuned based on the DINOv2 self-supervised learning method is inputted, the global features and local features of each color block are extracted through the tile embedding block and 24 Transformer blocks, and then the normalized output is the N x 1024-dimensional color block level feature matrix of the WSI image.

[0008] Further preferably, the first package level model is composed of an initialization full connection layer, a gated attention layer and a classifier layer; The initialization full connection layer is used to reduce the dimension of the original color block level features and cache the first color block descriptor matrix; The gated attention layer generates a first color block attention weight through a Tanh activation function and a Sigmoid activation function and is normalized by Softmax; The classifier layer outputs the first WSI package level feature matrix by multiplying the normalized first color block attention weight and the color block descriptor matrix point by point, and aggregates the first global feature for image level benign and malignant prediction.

[0009] The second package level model is composed of an initialization full connection layer, a gated attention layer and a classifier layer; The initialization full connection layer is used for dimension reduction of the first WSI package level feature matrix, and caches the second color block descriptor matrix; The gated attention layer generates the second color block attention weight through the Tanh activation function and the Sigmoid activation function and is normalized by the Softmax function; The classifier layer outputs the second WSI package level feature matrix by multiplying the normalized second color block attention weight and the second color block descriptor matrix point by point, and aggregates the second global feature for image level gene mutation prediction.

[0010] Further preferably, when training the second package level model, the parameters of the first package level model are connected in series with the second package level model, and the first package level model undertakes the screening function; if the predicted label is benign, the result is directly output; if it is malignant, the corrected feature matrix is output to the second package level model for gene mutation prediction.

[0011] Further preferably, removing invalid color blocks includes: filtering out color blocks in unorganized areas, empty package cavity areas and background areas by color convolution threshold setting.

[0012] In a second aspect, the application provides a device for predicting gene mutation based on WSI image cascade package level learning, comprising: A paraffin HE staining section digitization module is used for digitizing paraffin HE staining sections into WSI images; An image cutting quality control module cuts WSI images into color blocks according to maximum resolution equidistant, removes invalid color blocks, and obtains a color block set; A color block level feature extraction module uses a ViT model fine-tuned based on DINOv2 self-supervised learning method to extract features of the color block set of the WSI image, and generates a color block level feature matrix; the color block level feature matrix of a WSI image is regarded as a package; The cascade package level learning module is used for connecting a first package level model and a second package level model, assigning benign and malignant as a package level label to each package, training the first package level model based on the benign and malignant package level label and the color block level feature, obtaining a first color block attention weight through the first package level model, multiplying the first color block attention weight with a first color block descriptor matrix of the WSI image point by point, and outputting a first WSI package level feature matrix; assigning a gene mutation state as a package level label to each package, and training the second package level model based on the gene mutation state package level label and the first WSI package level feature matrix, wherein the second package level model is used for identifying a gene mutation state of a tumor region of the WSI image.

[0013] In a third aspect, the present application provides an electronic device, wherein the computer device comprises a data processor, a professional image processor, and a memory storing a visualization operating system, and the computer program is executed by the processor to implement the method for predicting a gene mutation based on a WSI image cascade package level learning.

[0014] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium is used to implement the method for predicting a gene mutation based on a WSI image cascade package level learning.

[0015] The present application has the following beneficial effects: The cascade package level model and the ViT model formed by the first package level model and the second package level model can effectively convert image information in the WSI into a feature matrix for package level learning, which provides a practical and theoretical basis for predicting a gene mutation state based on WSI image information. The ViT model is combined with the package level learning method, and the cascade mode is used to further improve the prediction performance of the gene mutation model, so as to realize the prediction of a gene mutation only through a WSI image, and to realize rapid pathological diagnosis in surgery, to quickly screen out patients with gene mutations, and to improve the efficiency and success rate of clinical trials.

[0016] The cascade package level model is used for two-step focusing of "rough to fine". The first color block attention weight of the first package level model compresses non-tumor color blocks to close to 0, and the second package level model re-distributes the second color block attention weight, and only the tumor color block is finely sorted. Compared with the pure package level model, the normal tissue interference is weakened, the feature signal noise ratio is significantly improved, the model performance is significantly improved, the color block attention weight map is more concentrated, the clinical interpretability is better, the training difficulty is decomposed, the overfitting risk is reduced, and any downstream mutation can be compatible. The algorithm migration cost is low. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The method flowchart of the present application.

[0018] Figure 2A cascade bag-level model structure diagram composed of a first-level bag-level model and a second-level bag-level model.

[0019] Figure 3 A first-level bag-level model ROC curve.

[0020] Figure 4 A second-level bag-level model ROC curve.

[0021] Figure 5 A single model ROC curve.

[0022] Figure 6 A first-level color block attention weight map output by the first-level bag-level model.

[0023] Figure 7 A second-level color block attention weight map output by the second-level bag-level model. DETAILED DESCRIPTION

[0024] In order to enable the above-mentioned objects, features and advantages of the present application to be more clearly understood, the present application will be described in further detail below with reference to embodiments. The specific embodiments described herein are merely intended to explain the present application and are not intended to limit the present application.

[0025] As shown in Figure 1 and Figure 2 , the embodiment provides a method for predicting gene mutation based on WSI image cascade bag-level learning, and the steps are as follows: Step one, obtaining WSI image: obtaining any disease paraffin HE staining section, removing paraffin HE staining sections without tissue, tissue damage overlap, and serious staining pollution, scanning the paraffin HE staining section into a WSI image (panoramic pathology scanning image), and the image format should be svs, tif, tiff, kfb, png, jpg; Step two, WSI image cutting and quality control: cutting the WSI image into color blocks with an actual physical distance of 128μm*128μm according to the maximum resolution, and filtering out invalid color blocks such as non-tissue area, empty bag cavity area, background area, etc. by setting a threshold through color convolution, to obtain a color block set; Step three: after color standardization processing of N color blocks of a WSI image, uniforming the color block size, then inputting a ViT model (Vision Transformer) fine-tuned based on a DINOv2 self-supervised learning method, extracting global features and local features of each color block through a patch embedding block (PatchEmbedding) and 24 Transformer blocks, and then performing normalization, and finally outputting an N*1024-dimensional color block level feature matrix of the WSI image, regarding the color block level feature matrix of a WSI image as a bag; Step four: benign and malignant are assigned to each package as a package level label, and a first-level package level model is trained based on the benign and malignant package level label and the color block level features. The first-level package level model obtains a first-level color block attention weight, and the first-level color block attention weight is multiplied point by point with the first-level color block descriptor matrix of the WSI image to output a first-level WSI package level feature matrix. The first-level package level model undertakes the functions of tumor discrimination and ROI region contribution calculation, is used for identifying the benign and malignant states of the WSI image, and can output the coordinates of the high malignant degree region and the first-level WSI package level feature matrix. The first-level package level model first judges the prediction label. If it is a benign tumor, the next step is not performed, and a benign label is directly output. If it is a malignant label, the first-level WSI package level feature matrix is output to a second-level package level model. Step five: the gene mutation state is assigned to each package as a package level label, and a second-level package level model is trained based on the gene mutation state package level label and the first-level WSI package level feature matrix. The second-level package level model is used for identifying the gene mutation state of the tumor region of the WSI image.

[0026] Further, in step one, the collected paraffin HE stained sections of breast tissue are different, the pathological state is clear, there is no damage, no folding, the image is clear, there is no staining pollution and impurities, and the digital processing is performed by a section scanner, and is saved as svs, tif, png and the like. In this embodiment, the target is set to establish a classification model capable of predicting the PIK3CA gene mutation state of the WSI image of breast cancer, therefore, the TCGA paraffin HE stained sections of breast cancer and the corresponding gene mutation labels are downloaded, the sections without tissue, tissue damage overlap, and severe staining pollution are excluded, a total of 640 examples (all malignant tumors), of which 575 examples have corresponding gene mutation labels. The paraffin HE stained sections of breast cancer in the hospital are collected and scanned into WSI images, the image format should be svs, the maximum resolution is 40x, a total of 311 examples, of which 144 examples are malignant tumors and 167 examples are non-malignant tumors.

[0027] Further, in step two, the WSI image is subjected to background removal processing to retain the region image where the tissue exists in the WSI image, and then subjected to quality control after being divided into color blocks with the same physical distance according to the highest resolution, and spatial position information is assigned to each color block to obtain a color block set. In this embodiment, the WSI image is divided into color blocks with an actual physical distance of 128 μm x 128 μm according to the maximum resolution of the WSI image, and the cut color blocks are subjected to quality control. The quality control content includes: 1. Background removal: set the image three-channel threshold, after color inversion, the overall average gray value is between 10-240, the three-channel variance mean is greater than 280, and the median gray value is also between 10-240, the color blocks with pure white, pure black or close to pure white and pure black are removed; 2. Size control: the cutting size is determined with reference to the maximum resolution of the WSI image, it is considered that “40x” resolution uses 512 pixels x 512 pixels, and “20x” uses 256 pixels x 256 pixels, the size of the divided color blocks is removed; 3. Clarity control: it is generally considered that when the clarity threshold is set to 150, the texture and color of the color block are still meaningful, and the color blocks with a clarity lower than 150 are removed, and finally the color block set is obtained.

[0028] Further, in step three, after the color block set is subjected to color standardization processing, the size is unified to 224 pixels x 224 pixels x 3 channel size image matrix, the batch size is set to 128, the ViT model (Vision Transformer) is input, the ViT model used in this example is the UNI pathology base model disclosed by the mahmood laboratory, which is trained based on the DINOv2 self-supervised learning method. The specific principle is that the ViT model learns the image-level target and patch-level target through the cross-entropy loss function and the distillation loss function in the form of random mask and class label, so as to learn the global feature and local feature. After 1 patch embedding and 24 transformer blocks, the depth feature of each color block is extracted, the features are normalized after all the transformer blocks, and a head is used for final output to obtain an N x 1024-dimensional color block level feature matrix of the WSI image. The color block level feature matrix of a WSI image is regarded as a bag.

[0029] Further, in step four, 144 malignant tumor packages and 167 non-malignant tumor packages of the hospital-derived WSI images and 65 randomly selected malignant tumor packages of the TCGA-derived WSI images are used as the training set. The random number is set to 123, and 10 internal training sets, test sets, and validation sets are randomly constructed in the ratio of 8:1:1, i.e., ten-fold cross-validation. The maximum loop is set to 200, the learning rate is set to 0.0001, the L2 regularization first-level color block attention weight decay coefficient is set to 0.00001, the positive and negative instance sample collection limit is set to 8, and the ratio of the package-level loss to the color block loss in the total loss is set to 7:3. The first-level package-level model is composed of three parts: an initialized full connection layer, a gated attention layer, and a classifier layer. The initialized full connection layer first reduces the received original N x 1024-dimensional color block level features to N x 512, while buffering the N x 1024-dimensional first-level color block descriptor matrix; the gated attention layer generates N x 256 attention through two activation functions "Tanh branch" and "Sigmoid branch", respectively, point multiplies and fully connects to obtain N x 1 original attention score, and then normalizes it to the first-level color block attention weight α1 using Softmax; the classifier layer multiplies the first-level color block attention weight α1 and the N x 1024-dimensional first-level color block descriptor matrix point by point to output the N x 1024-dimensional first-level WSI package-level feature matrix, and then adds the same columns to output the 1 x 1024 first-level global feature. Finally, the classifier layer gives the image-level benign and malignant prediction, and outputs the predicted benign and malignant label. The performance of multiple candidate models is evaluated through ten-fold cross-validation, and the AUC-ROC values (area under the working characteristic curve) obtained by cross-validation of each model are compared, as shown in Figure 3 . Figure 3 The sensitivity represents the proportion of correctly identified malignant (or mutant) cases by the model, the 1-specificity represents the proportion of incorrectly identified benign (or non-mutant) cases as malignant (or mutant) by the model, the AUC value is the area under the ROC curve, the value range is 0~1, the CI is the 95% confidence interval, and the model with the optimal (usually the highest) AUC-ROC value is finally selected, Figure 3 The AUC-ROC value of model 5 is 0.996, so the performance of model 5 is optimal, and model 5 is used as the optimal first-level package-level model. The first-level color block attention weight map of the first-level package-level model is shown in Figure 6 .

[0030] Further, in step five, 575 TCGA-derived malignant tumor packages with corresponding gene mutation labels are taken as a training set, of which 176 packages are packages with PIK3CA gene mutations, and 399 packages are packages without PIK3CA gene mutations. The 575 TCGA-derived packages are input into the optimal primary package level model, and the output primary WSI package level feature matrix is input into the secondary package level model for training. The random number is set to 123, and 10 internal training sets, test sets and validation sets are randomly constructed in a ratio of 8:1:1; the maximum cycle is set to 200, the learning rate is set to 0.0001, the L2 regularization secondary tile attention weight decay coefficient is set to 0.00001, the positive and negative instance sample collection limit is set to 8, and the ratio of package level loss to tile loss in the total loss is set to 8:2. The secondary package level model is initialized based on the parameters of the optimal primary package level model, so that the secondary package level model inherits the ability of the optimal primary package level model to extract pathological features. The secondary package level model is composed of three parts: an initialized full connection layer, a gated attention layer and a classifier layer. The initialized full connection layer reduces the primary WSI package level feature matrix output by the optimal primary package level model to N x 512, and buffers an N x 1024-dimensional secondary tile descriptor matrix; the gated attention layer generates N x 256 attention through two activation functions "Tanh branch" and "Sigmoid branch" respectively, obtains N x 1 original attention score through point multiplication and full connection, and normalizes the secondary tile attention weight a2 through Softmax. The classifier layer multiplies the secondary tile attention weight a2 and the N x 1024-dimensional secondary tile descriptor matrix point by point to output an N x 1024-dimensional secondary WSI package level feature matrix, and outputs a 1 x 1024 secondary global feature by column addition. Finally, the classifier layer gives a gene mutation prediction at the image level, and outputs the gene mutation label. The performance of multiple candidate models is evaluated through ten-fold cross-validation. Model 3 has the best performance, and is taken as the optimal secondary package level model. The secondary tile attention weight map of the secondary package level model is shown in Figure 7 After the first correction, the attention drop point is more concentrated in the highly malignant area. The AUC-ROC value (area under the working characteristic curve) of model is 0.711 Figure 4 , which is 6% higher than the AUC-ROC value 0.65 of the optimal model 5 of the independent gene mutation prediction model Figure 5 .

[0031] Drawing the spatial position of the tile and the global tile attention weight of different gene mutation states can find that the weakly supervised attention drop point is located in the tumor position of the WSI image, which is consistent with the laboratory detection judgment logic, so it is believed that the method has certain explainability and can perform preliminary gene mutation state discrimination.

[0032] The application uses a disclosed general big data image model, which has the advantages of large model structure depth, strong feature extraction capability, and strong robustness suitable for most pathological images.

[0033] In one embodiment, the device for predicting gene mutation based on WSI image cascade package level learning includes a paraffin HE staining section digitization module, an image cutting quality control module, a color block level feature extraction module, and a cascade package level learning module. The paraffin HE staining section digitization module is used to digitize paraffin HE staining sections into WSI images in svs, tif, tiff, kfb, png, and jpg formats; The image cutting quality control module is used to cut WSI images into color blocks with an actual physical distance of 128 μm*128 μm, filter out invalid color blocks such as non-tissue areas, empty package cavity areas, and background areas, assign spatial position labels to each valid color block, and obtain a color block set; The color block level feature extraction module uses a ViT model fine-tuned based on a DINOv2 self-supervised learning method to extract features of the color block set of the WSI image and generate a color block level feature matrix; and a color block level feature matrix of a WSI image is regarded as a package; The cascade package level learning module connects a first package level model and a second package level model, assigns benign and malignant as package level labels to each package, trains the first package level model based on the benign and malignant package level labels and the color block level features, obtains first color block attention weights through the first package level model, and outputs a first WSI package level feature matrix after the first color block attention weights are multiplied point by point with a first color block descriptor matrix of the WSI image; and the second package level model is used to identify the gene mutation state of the tumor region of the WSI image.

[0034] In one embodiment, a computer device includes a data processor, a professional image processor, a memory storing a visualization operating system, and a computer program executed by the processor to perform the prediction method of any one of the above.

[0035] In one embodiment, a computer readable storage medium is used to perform the prediction method of any one of the above.

[0036] Obviously, the above embodiments of the present application are merely exemplary but not intended to limit the embodiments of the present application. Based on the above description, any other variations or changes can be made by those skilled in the art without departing from the spirit and principles of the present application. It is not necessary to list all the embodiments here. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application should be included in the protection scope of the claims of the present application.

Claims

1. A method for predicting gene mutations based on WSI image cascade bag-level learning, characterized in that, The application relates to a method for predicting the benign and malignant states of a tumor region in a whole slide image (WSI) based on a WSI image. The method comprises the following steps: Obtaining a paraffin HE-stained section of any disease and scanning the section into a WSI image; Cutting the WSI image into patches according to the maximum resolution, removing invalid patches, and obtaining a patch set; Using a ViT model fine-tuned based on a DINOv2 self-supervised learning method to extract features of the patch set of the WSI image, and generating a patch-level feature matrix; Regarding the patch-level feature matrix of a WSI image as a bag; Assigning the benign and malignant states as bag-level labels to each bag, training a primary bag-level model based on the benign and malignant bag-level labels and the patch-level features, obtaining primary patch attention weights through the primary bag-level model, multiplying the primary patch attention weights with a primary patch descriptor matrix of the WSI image point by point, and then outputting a primary WSI bag-level feature matrix, and outputting an image-level benign and malignant prediction by the primary bag-level model; 2. The method of claim 1, wherein, Assigning a gene mutation state as a bag-level label to each bag, and training a secondary bag-level model based on the gene mutation state bag-level label and the primary WSI bag-level feature matrix, wherein the secondary bag-level model is used for identifying the gene mutation state of a tumor region in a WSI image.

3. The method of claim 1, wherein, The feature extraction process is as follows: after color normalization of N patches of a WSI image, the patch size is unified, and then the patches are input into a ViT model fine-tuned based on a DINOv2 self-supervised learning method; global features and local features of each patch are extracted through a patch embedding block and 24 Transformer blocks; and finally, the normalized output is an N*1024-dimensional patch-level feature matrix of the WSI image. The primary bag-level model comprises an initialization full connection layer, a gated attention layer and a classifier layer; The initialization full connection layer is used for dimension reduction of original patch-level features and caching of a primary patch descriptor matrix; The gated attention layer generates primary patch attention weights through a Tanh activation function and a Sigmoid activation function and performs Softmax normalization; 4. The method of claim 1, wherein, The classifier layer multiplies the normalized primary patch attention weights with the patch descriptor matrix point by point, outputs a primary WSI bag-level feature matrix, and aggregates the primary WSI bag-level feature matrix into a primary global feature for image-level benign and malignant prediction. The secondary bag-level model comprises an initialization full connection layer, a gated attention layer and a classifier layer; The initialization full connection layer is used for dimension reduction of the primary WSI bag-level feature matrix and caching of a secondary patch descriptor matrix; The gated attention layer generates secondary patch attention weights through a Tanh activation function and a Sigmoid activation function and performs Softmax normalization; 5. The method of claim 4, wherein, The classifier layer multiplies the normalized secondary patch attention weights with the secondary patch descriptor matrix point by point, outputs a secondary WSI bag-level feature matrix, and aggregates the secondary WSI bag-level feature matrix into a secondary global feature for image-level gene mutation prediction. During training of the secondary bag-level model, the parameters of the primary bag-level model are connected in series with the secondary bag-level model, and the primary bag-level model undertakes a screening function; if the prediction label is benign, the result is directly outputted; If the prediction label is malignant, the corrected feature matrix is outputted to the secondary bag-level model for gene mutation prediction.

6. The method of claim 1, wherein, The invalid color blocks are removed, including: filtering out color blocks of unorganized areas, empty package cavity areas and background areas by color convolution threshold setting.

7. A device for predicting gene mutations based on WSI image cascaded package level learning, characterized in that, It comprises: A paraffin HE staining section digitization module is configured to digitize a paraffin HE staining section into a WSI image. An image cutting quality control module is configured to cut the WSI image into color blocks according to maximum resolution equidistance, remove invalid color blocks, and obtain a color block set. A color block level feature extraction module is configured to use a ViT model fine-tuned based on a DINOv2 self-supervised learning method to extract features of the color block set of the WSI image, and generate a color block level feature matrix. A WSI image color block level feature matrix is regarded as a package. A cascaded package level learning module is configured to connect a first-level package level model and a second-level package level model, assign benign and malignant as package level labels to each package, train the first-level package level model based on the benign and malignant package level labels and the color block level features, obtain first-level color block attention weights through the first-level package level model, multiply the first-level color block attention weights with a first-level color block descriptor matrix of the WSI image point by point, and output a first-level WSI package level feature matrix. The genetic mutation state is assigned as a package level label to each package, and a second-level package level model is trained based on the genetic mutation state package level label and the first-level WSI package level feature matrix, and the second-level package level model is used to identify the genetic mutation state of the tumor area of the WSI image.

8. An electronic device, comprising: The computer device comprises a data processor and a professional image processor, the memory stores a visualization operating system, and the computer program is executed by the processor to execute the method of any one of claims 1-6.

9. An apparatus readable storage medium characterized by, The computer readable storage medium is used to execute the method of any one of claims 1-6.

Images