A cancer survival prediction method based on a visual language model

Abstract

The present application relates to the technical field of medical image analysis and computational pathology, and proposes a cancer survival prediction method based on a visual language model. The method first acquires a pathological whole field slice image of a cancer patient, detects the tissue area of the slice and divides it into multiple image blocks, and extracts visual features of the image blocks through a pre-trained visual encoder. Then, a metastasis diagnosis model is used to predict the image blocks, generate metastasis pseudo-labels and uncertainty information at the image block level, and obtain metastasis priors. On this basis, an adaptive hierarchical prompting mechanism is constructed, including slice-level survival semantic prompts and image block-level metastasis semantic prompts, and corresponding text semantic features are generated through a visual language model text encoder. Subsequently, the image block visual features, metastasis prior information and text semantic features are combined to generate slice-level representation features through a concept-guided feature aggregation module. Finally, the slice-level representation is optimized using an uncertainty perception calibration strategy, and the survival risk prediction result of the cancer patient is output according to the calibrated features. The present application can effectively utilize the multi-scale tissue morphology information and metastasis-related semantic information in the pathological whole field slice under the condition of few samples, realize the automatic prediction of cancer survival risk, and can be applied to a computer-aided pathological diagnosis and prognosis evaluation system.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis and computational pathology, and in particular to a cancer survival prediction method based on a vision-language model. Specifically, it relates to a computer implementation method for modeling metastasis-related features and predicting cancer survival using whole-slide images (WSI) under conditions of limited sample size. Background Technology

[0002] Cancer prognostic assessment is a crucial aspect of clinical oncology, playing a vital role in developing individualized treatment plans, assessing disease progression risk, and guiding clinical decision-making. With the advancement of digital pathology technology, whole-slide images (WSI) can record tissue morphology and tumor microenvironment information at extremely high resolution, providing an important data foundation for automated prognostic analysis using computational methods.

[0003] In computational pathology, deep learning-based methods have been widely applied to tasks such as tumor subtyping, tissue structure recognition, biomarker discovery, and survival prediction. Due to the ultra-high resolution and complex spatial structure of WSI images, researchers typically divide the entire slice into multiple patches and perform feature learning using weakly supervised learning frameworks. Among these, the Multiple Instance Learning (MIL) method, which treats the entire slice as a set of multiple patches, achieves feature aggregation and prediction using only slice-level labels, and has been widely used in WSI analysis tasks.

[0004] However, compared to traditional classification tasks, cancer survival prediction requires modeling time-related survival outcomes, making it a more complex task. Existing methods typically rely on large amounts of labeled data for training, but high-quality survival follow-up data is often scarce in real-world clinical settings, limiting the model's generalization ability in low-sample scenarios.

[0005] On the other hand, tumor metastasis is one of the leading causes of death in cancer patients and a key factor affecting patient prognosis. The metastasis process involves the spread of tumor cells to distant tissues and exhibits complex and subtle morphological features at the histological level. However, in existing survival prediction methods based on WSI (Warranty-Survey Indicator), most models mainly focus on the overall morphological features of the tumor region, paying insufficient attention to local tissue patterns related to metastasis, thus making it difficult to fully utilize pathological information closely related to prognosis.

[0006] In recent years, Vision-Language Models (VLMs) have demonstrated strong capabilities in multimodal representation learning by jointly learning image information and textual semantic information, and have gradually gained attention in the field of medical image analysis. By aligning pathological images with medical concepts or diagnostic text, VLMs can achieve good knowledge transfer capabilities under conditions of few or no samples. However, the application of existing VLMs in pathological WSI survival prediction tasks still has certain limitations. For example, some methods only use simple text prompts to guide model learning, which is insufficient to characterize the hierarchical semantic information involved in the pathological diagnosis process; at the same time, existing feature aggregation methods often rely mainly on visual features themselves, lacking clear clinical semantic concept guidance, resulting in the generated slice-level representations failing to fully reflect key pathological information related to prognosis.

[0007] Therefore, how to fully utilize the multi-level features in the full field of pathological sections under limited sample conditions, and combine them with clinical semantic information related to tumor metastasis to improve the accuracy and interpretability of cancer survival prediction, remains a technical problem that urgently needs to be solved in the field of computational pathology. Summary of the Invention

[0008] To address the shortcomings of existing cancer survival prediction methods based on whole-slide images (WSI) in limited-sample conditions, such as unstable prediction performance, difficulty in effectively utilizing pathological information related to tumor metastasis, and lack of joint modeling capability for multi-scale tissue structures, this invention provides a cancer survival prediction method based on a vision-language model. This method introduces prior information about metastasis, a hierarchical semantic cue mechanism, and an uncertainty-aware calibration strategy to collaboratively model pathological images and medical semantic concepts in a vision-language joint representation space. This enables accurate prediction of cancer patient survival status and improves the model's generalization ability and prediction reliability under limited-sample conditions.

[0009] This invention introduces clinical prognostic knowledge (including survival stratification and pathological concepts related to tumor metastasis) into the semantic space of a visual-language model. This allows the model to simultaneously utilize visual features from pathological images and semantic knowledge from medical concepts, thus overcoming the information deficiency inherent in traditional visual models that rely solely on image features for learning. Furthermore, this invention utilizes prior information about metastasis to provide auxiliary supervision of local tissue regions and incorporates an uncertainty estimation mechanism to calibrate prediction results, thereby reducing the impact of pseudo-label noise on model performance under weakly supervised learning conditions.

[0010] To achieve the above objectives, this invention proposes a cancer survival prediction method based on visual-language modeling, which includes the following steps.

[0011] First, pathological full-view slice image data of cancer patients is acquired, and the images are preprocessed. Since WSI images typically have extremely high resolution, with pixel counts often reaching billions, they cannot be directly used as the entire image input for deep learning model computation. This invention first performs tissue region detection on the WSI images to remove background regions. Then, the remaining tissue regions are divided into multiple non-overlapping patches of a preset size, resulting in a set of patches. Assuming each WSI image is divided into k patches, it can be represented as:

[0012] .

[0013] Where, x i Let represent the i-th tile. Using the above method, the original ultra-large-scale WSI image can be transformed into a sequence of tiles that can be processed by deep learning models.

[0014] Secondly, this invention constructs a metastasis prior acquisition module to obtain local supervision information related to tumor metastasis. In real clinical datasets, survival prediction data often only contain patient-level or slice-level survival labels, lacking annotation information on the presence of metastatic lesions in local tissue regions. However, tumor metastasis is one of the key factors affecting the survival prognosis of cancer patients; therefore, introducing metastasis-related information during model training is of great significance for improving survival prediction performance.

[0015] To this end, this invention trains a diagnostic network on an auxiliary dataset with explicit transfer labels to identify the presence of transfer tissue in map tiles. After training, the diagnostic network is applied to maps in the target WSI dataset to generate a transfer probability prediction result for each map tile. The prediction result can be expressed as:

[0016]

[0017] in, The predicted label indicates whether the i-th patch contains the transferred tissue. This predicted label serves as prior information about the transfer and provides local-level auxiliary supervision signals for subsequent model training.

[0018] Considering the potential noise in pseudo-labels generated by the auxiliary model, this invention further introduces an uncertainty estimation mechanism to assess the reliability of the transfer prediction results. Specifically, a random dropout mechanism is enabled during the model inference phase, and multiple random forward propagations are performed on each tile to obtain multiple sets of prediction results. By calculating the variance between these prediction results, the transfer prediction uncertainty index for each WSI can be obtained. This index reflects the model's confidence in identifying the transfer region and provides a basis for calibrating subsequent prediction results. The slice-level uncertainty calculation method is as follows:

[0019]

[0020] Where k represents the number of tiles in a single slice, m represents the number of forward propagations, and y i Let E be the label of the i-th image tile, E represent the expectation, and U represent the uncertainty of the final result.

[0021] After obtaining tile features and metastasis prior information, this invention constructs an adaptive hierarchical prompting module to effectively adapt the visual-language model to cancer prognostic tasks. By constructing slice-level and tile-level prompts, the visual-language model can simultaneously encode global prognostic information and local pathological information. Specifically, at the slice-level, this invention constructs semantic prompts representing different survival states or disease severity to reflect the overall prognostic level of patients; at the tile-level, this invention constructs semantic prompts representing local tissue states, such as the presence or absence of metastatic tissue.

[0022] Furthermore, this invention introduces learnable context vectors into semantic prompts. By optimizing these vectors during training, the prompts can dynamically adapt to specific task requirements, thereby enhancing the visual-language model's ability to express medical semantic information. In this way, the model can better adapt to cancer survival prediction tasks while maintaining its original cross-modal representation capabilities.

[0023] After constructing semantic prompts, this invention further proposes a concept-guided feature aggregation module to generate clinically meaningful slice-level representation features. Because significant morphological differences exist between different tissue regions in WSI images, and key pathological features related to survival prognosis often only appear in local areas, simple average pooling or max pooling methods are insufficient to effectively capture this important information.

[0024] To this end, this invention first utilizes an image encoder in a vision-language model to extract features from each patch, thereby obtaining a visual feature vector for the patch. Subsequently, guided modeling of the patch features is performed by introducing semantic anchors. These semantic anchors are used to represent typical metastasis-related pathological patterns and interact with the patch features through a cross-modal attention mechanism, thereby highlighting important regional features related to metastasis.

[0025] After enhancing local features, this invention uses an attention mechanism to aggregate all patch features, thereby generating a slice-level feature representation. Compared with traditional feature aggregation methods, the concept-guided aggregation method proposed in this invention can introduce medical semantic knowledge into the feature aggregation process, so that the final slice-level representation not only contains visual information but also integrates medical concepts related to prognosis. The formula is as follows:

[0026]

[0027]

[0028]

[0029] Where X represents the patch feature. V represents the intermediate patch feature, and t represents the intermediate slice feature. p With t s Text embedding at the tile and slice levels, respectively, A p With A s W represents a learnable context vector. Q W K W V , , , , , represents the weight parameters of the projection layer, cat indicates the concatenation operation, V is the final slice-level feature obtained, and d is the dimension.

[0030] After obtaining slice-level representation features, this invention further guides the feature space using slice-level semantic concepts, enabling the model to explicitly encode survival-related semantic information in the feature representation. By aligning slice-level visual features with textual semantic features, the similarity between different survival categories can be obtained, thereby achieving survival prediction.

[0031] Considering that model training under weak supervision is easily affected by noisy samples, this invention also proposes an uncertainty-aware calibration module to adjust the global prediction results using local transfer information. Specifically, this invention first sorts all tiles according to the similarity between the tiles and the transfer-related semantic concepts, and selects a subset of tiles with the highest confidence as key regions. Subsequently, the prediction information of these key regions is aggregated and fused with the slice-level prediction results.

[0032] During the fusion process, this invention utilizes the uncertainty index of transfer prediction to perform temperature smoothing on the prediction results, thereby avoiding excessive reliance on unreliable local information. When the uncertainty of transfer prediction is high, the model automatically reduces the influence of local information on the final prediction result; when the uncertainty is low, the model enhances the contribution of local transfer information to the prediction result. In this way, transfer-related pathological information can be fully utilized while ensuring prediction stability. The model's loss function is:

[0033]

[0034]

[0035] in, p is a hyperparameter, q is a hyperparameter balancing noise immunity and convergence, and p is a hyperparameter. y It is the predicted probability of label y. For slice-level visual-language alignment contrast loss, The optimization objective is for tile-level pseudo-labels.

[0036] Through the above steps, the present invention can achieve collaborative modeling of multi-scale pathological information and medical semantic concepts in the visual-linguistic joint representation space, thereby generating clinically meaningful slice-level representations and ultimately outputting cancer survival prediction results.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] First, by introducing prior information about metastasis, this invention enables the model to automatically focus on key pathological regions related to tumor metastasis, thereby improving the clinical relevance of survival prediction results.

[0039] Second, by designing an adaptive hierarchical prompting mechanism, this invention enables the visual-language model to simultaneously encode slice-level survival semantic information and tile-level transition semantic information, thereby improving the model's ability to model multi-scale pathological features.

[0040] Third, by proposing a concept-guided feature aggregation method, this invention enables slice-level representations to not only include visual structural information but also integrate medical semantic knowledge, thereby improving the interpretability of model prediction results.

[0041] Fourth, by introducing an uncertainty-aware calibration mechanism, this invention effectively reduces the impact of pseudo-label noise on model training under weak supervision, thereby improving the model's prediction stability and generalization ability under conditions of limited sample data.

[0042] In summary, this invention organically combines visual-language models, multi-scale pathological image features, and medical semantic knowledge to realize a cancer survival prediction method suitable for small sample conditions. It can improve the robustness and interpretability of the model while ensuring prediction accuracy, and has important application value in the fields of computational pathology and intelligent medicine. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the overall framework of the cancer survival prediction method described in this invention;

[0044] Figure 2 This is a schematic diagram of the uncertainty perception calibration module described in this invention;

[0045] Figure 3 This is a schematic diagram of the adaptive hierarchical prompting module described in this invention;

[0046] Figure 4 This is a schematic diagram of the structure of the concept-guided feature aggregation module described in this invention. Detailed Implementation

[0047] The cancer survival prediction method based on vision-language modeling proposed in this invention will be described in detail below with reference to specific embodiments. These embodiments are only used to illustrate the technical solution of this invention and are not intended to limit the scope of protection of this invention. Those skilled in the art can make various improvements or equivalent substitutions to the implementation methods without departing from the spirit of this invention.

[0048] This invention proposes a cancer survival prediction method based on a vision-language model. Its overall technical framework mainly includes the following modules: a pathological image preprocessing module, a metastasis prior acquisition module, an adaptive hierarchical prompting module, a concept-guided feature aggregation module, and an uncertainty perception calibration module. Through the collaborative work of these modules, accurate prediction of cancer patient survival status can be achieved even with limited sample sizes.

[0049] 1. System Overall Process

[0050] The overall process of the method proposed in this invention is as follows: Figure 1 As shown, the details are as follows:

[0051] First, pathological full-view slice image data of cancer patients are acquired and preprocessed to divide the tissue region into multiple patches. Then, a transfer prior acquisition module generates patch-level transfer probability information. Next, a visual-language model is used to encode patch features and semantic cues, and a hierarchical cues mechanism is used to establish the connection between image features and medical semantic concepts. Then, a concept-guided feature aggregation module fuses patch features to generate slice-level feature representations. Finally, an uncertainty perception calibration module adjusts the prediction results to obtain cancer survival prediction results.

[0052] 2 Data Acquisition and Preprocessing

[0053] In this embodiment, the experimental data were obtained from the publicly available Cancer Genome Atlas (TCGA) database. Specifically, the Breast Invasive Carcinoma (BRCA) dataset was used, which contains full-field pathological slide images and corresponding survival follow-up information of 1022 patients.

[0054] To evaluate the model's generalization ability, this embodiment employs a five-fold cross-validation strategy to partition the dataset. Specifically, all samples are randomly divided into five subsets. Each time, one subset is selected as the test set, and the remaining four subsets are used as the training set. This process is repeated five times, and the results of the five experiments are averaged.

[0055] Since WSI images typically have extremely high resolution, directly inputting them into a neural network for training would incur a huge computational cost; therefore, preprocessing of WSI images is necessary. First, a tissue region detection algorithm is used to segment the WSI image to remove background regions. Specifically, the CLAM method is employed to extract tissue regions from WSI images at 20x magnification, thereby obtaining the effective tissue regions.

[0056] Subsequently, the extracted tissue region is divided into multiple non-overlapping tiles of a fixed size. In this embodiment, each tile is 256×256 pixels. If each WSI is divided into k tiles, it can be represented as a tile set:

[0057] X = [x1, x2, …, xk]

[0058] Where xi represents the i-th tile.

[0059] To maintain consistency with the input format of the vision-language model, this embodiment performs standardization on each patch, including image resizing and pixel normalization. Subsequently, feature extraction is performed on each patch using an image encoder from the pre-trained vision-language model. In this embodiment, the ViT-B / 16 from the CLIP model is used as the image encoder, and its parameters are kept frozen. A 512-dimensional visual feature vector is extracted from each patch using the image encoder, and these feature vectors are stored for subsequent model training.

[0060] 3. Transfer Prior Acquisition Module

[0061] Tumor metastasis is a crucial factor influencing the prognosis of cancer patients. However, most survival prediction datasets typically only provide patient-level or slice-level survival labels, lacking patch-level metastasis annotation information. Therefore, this invention introduces an auxiliary dataset to construct prior metastasis information.

[0062] In this embodiment, the PatchCamelyon (PCam) dataset is used as auxiliary training data. This dataset contains pathological images of lymph node tissue and labels each patch to indicate whether it contains metastatic tissue. The specific implementation process is as follows: Figure 2 As shown.

[0063] First, 5000 images were randomly selected from the PCam dataset as training samples. Since the images in the PCam dataset are 96×96 pixels, while the input size of the CLIP model is 224×224 pixels, the images need to be resized. Specifically, the original images were resized from 96×96 to 224×224.

[0064] Subsequently, a lightweight diagnostic network M was trained using the aforementioned training data. D This network is used to identify the presence of metastatic tissue in a map patch. After training, it is applied to maps in the TCGA-BRCA dataset to generate a metastatic probability prediction for each patch.

[0065] Let the transition prediction result for the i-th tile be:

[0066] ŷi ∈ {0,1}

[0067] Where ŷi represents the prediction label for whether the patch contains a transgenic organization.

[0068] Since pseudo-labels may contain noise, this embodiment further estimates the prediction uncertainty using the Monte Carlo Dropout method. Specifically, the Dropout layer is kept on during the inference phase, and multiple random forward propagations are performed on the same tile to obtain multiple sets of prediction results. By calculating the variance of these prediction results, the confidence index of the model for the transition prediction can be obtained. The slice-level uncertainty is calculated as follows:

[0069]

[0070] Where k represents the number of tiles in a single slice, m represents the number of forward propagations, and y i Let E be the label of the i-th image tile, E represent the expectation, and U represent the uncertainty of the final result.

[0071] 4. Adaptive Hierarchical Hint Module

[0072] To enable the effective application of vision-language models in cancer survival prediction tasks, this invention proposes an adaptive hierarchical prompting mechanism, the specific structure of which is described in [link to specific structure]. Figure 3 By constructing multi-level semantic prompts, medical knowledge is integrated into the semantic space of the visual-language model.

[0073] At the slice level, this embodiment classifies WSI into four prognostic categories based on patient survival status and constructs corresponding natural language prompts. For example:

[0074] “A whole-slide image of low-grade breast cancer”

[0075] The above text description can reflect medical semantic information of different survival risk levels, thereby helping the model learn global prognostic features.

[0076] At the tile level, this embodiment constructs local semantic cues based on prior transition information. For example:

[0077] “showing metastatic tissue”

[0078] The above textual description can provide fine-grained semantic supervision information for patch features.

[0079] Furthermore, this embodiment introduces learnable context vectors to dynamically adjust the prompts. During training, these context vectors are continuously updated through backpropagation, enabling the prompts to gradually adapt to the specific task requirements.

[0080] To enable interaction between semantic information at different levels, this invention also designs a cross-scale cue fusion mechanism, which allows slice-level cue and tile-level cue to exchange information in the feature space, thereby enhancing the model's ability to model multi-scale pathological structures.

[0081] 5. Concept-guided feature aggregation module

[0082] After obtaining the visual features and semantic cue features of the tiles, this invention generates a slice-level representation through a concept-guided feature aggregation module, the specific results of which are as follows: Figure 4 As shown.

[0083] First, all patch features are input into a visual encoder to obtain a patch feature matrix. Then, a set of semantic anchors is introduced to represent typical metastasis-related pathological patterns. Through a cross-modal attention mechanism, these semantic anchors interact with the patch features, thereby highlighting important regions related to metastasis.

[0084] In this way, the representation of key pathological regions can be enhanced in the feature space, while suppressing interference from irrelevant regions.

[0085] After local feature enhancement, an attention mechanism is used to weightedly fuse all patch features to obtain a slice-level feature representation. Compared with traditional average pooling methods, this method can better capture important structural information in WSI.

[0086] Subsequently, this invention further introduces the concept of slice-level semantics, aligning slice-level visual features with textual semantic features through a cross-modal attention mechanism, thereby generating the final slice-level representation vector. The formula is as follows:

[0087]

[0088]

[0089]

[0090] Where X represents the patch feature. V represents the intermediate patch feature, and t represents the intermediate slice feature. p With t s Text embedding at the tile and slice levels, respectively, A p With A s W represents a learnable context vector. Q W K W V , , , , , represents the weight parameters of the projection layer, cat indicates the concatenation operation, V is the final slice-level feature obtained, and d is the dimension.

[0091] 6 Uncertainty Perception Calibration Module

[0092] Since the pseudo-labels generated in weakly supervised learning may contain noise, the prediction results need to be calibrated.

[0093] In this embodiment, all maps are first sorted according to the similarity between the map tiles and the transfer-related text concepts, and the K maps with the highest confidence are selected as key regions.

[0094] Subsequently, the prediction results for these key areas are averaged to obtain the local migration prediction vector.

[0095] To avoid the model becoming overly reliant on unreliable local information, this invention utilizes an uncertainty index in transition prediction to perform temperature smoothing on the prediction results. When uncertainty is high, the model reduces the impact of local information on the final prediction result; when uncertainty is low, the model increases the weight of local information.

[0096] In this way, relevant pathological information can be fully utilized while ensuring predictive stability. The model's loss function is:

[0097]

[0098]

[0099] in, p is a hyperparameter, q is a hyperparameter balancing noise immunity and convergence, and p is a hyperparameter. y It is the predicted probability of label y. For slice-level visual-language alignment contrast loss, The optimization goal is to achieve tile-level pseudo-labels.

[0100] 7. Prediction and Assessment

[0101] After the model is trained, it can be used to perform survival prediction on new WSI images.

[0102] First, the WSI image is preprocessed in the same way, and patch features are extracted; then, a slice-level representation is generated through a visual-language model; finally, the predicted probability of different survival categories is obtained through similarity calculation.

[0103] In this embodiment, model performance is evaluated using the Concordance Index (C-index). This index measures the accuracy of the model in ranking patient survival times. A higher C-index value indicates better model predictive performance.

[0104] All experimental results were statistically analyzed based on five-fold cross-validation, and the mean and standard deviation were reported to ensure the reliability of the experimental results.

Claims

1. A cancer survival prediction method based on a visual language model, characterized in that, Includes the following steps: Step 1: Obtain the full-view pathological section image of the cancer patient, perform tissue region detection on the full-view pathological section image and remove the background region, and divide the remaining tissue region into multiple patches; Step 2: Use a pre-trained visual encoder to extract features from the image patch to obtain the corresponding visual features of the image patch; Step 3: Construct a transition prior acquisition model to predict transitions of tiles and generate tile-level transition prior information; Step 4: Construct an adaptive hierarchical prompting mechanism, including slice-level semantic prompts and image block-level semantic prompts, and encode the semantic prompts through a visual language model to obtain text semantic features; Step 5: Based on the visual features of image patches, prior information about the transition, and semantic features of the text, generate slice-level representation features through the concept-guided feature aggregation module; Step 6: Based on the prior information of the tile-level transition and the prediction uncertainty, perform uncertainty-aware calibration on the slice-level representation features; Step 7: Calculate the cancer survival prediction result based on the similarity between the calibrated slice-level representation features and the text semantic features.

2. The method according to claim 1, characterized in that, In step 1, the tissue area is divided into fixed-size tiles, each tile being 256×256 pixels in size.

3. The method according to claim 1, characterized in that, In step 2, the pre-trained visual encoder is an image encoder in the visual language model, used to map patches into high-dimensional visual feature vectors.

4. The method according to claim 1, characterized in that, In step 3, the transfer prior acquisition model is trained on a dataset with transfer labels and used to predict whether each tile contains transfer tissue, thereby generating tile-level pseudo-labels.

5. The method according to claim 1, characterized in that, In step 4, the adaptive hierarchical prompting mechanism includes: constructing slice-level semantic prompts to describe different survival risk levels; constructing image patch-level semantic prompts to describe local tissue states; and introducing learnable context vectors to dynamically adjust the semantic prompts.

6. The method according to claim 1, characterized in that, In step 5, the visual features of image patches are weighted and fused through the concept-guided feature aggregation module, and key regions are enhanced through semantic anchors and cross-modal attention mechanisms to generate slice-level representation features, as shown in the following formula: Where X represents the patch feature. V represents the intermediate patch feature, and t represents the intermediate slice feature. p With t s Text embedding at the tile and slice levels, respectively, A p With A s W represents a learnable context vector. Q W K W V , , , , , represents the weight parameters of the projection layer, cat indicates the concatenation operation, V is the final slice-level feature obtained, and d is the dimension.

7. The method according to claim 1, characterized in that, In step 5, the transition prediction results are subjected to multiple random forward propagations using the Monte Carlo Dropout method to calculate the prediction variance and obtain the uncertainty index. The slice-level uncertainty is calculated as follows: Where k represents the number of tiles in a single slice, m represents the number of forward propagations, and y i Let E be the label of the i-th image tile, E represent the expectation, and U represent the uncertainty of the final result.

8. The method according to claim 1, characterized in that, Step 5 also includes sorting the tiles according to the similarity between the tiles and the semantic concepts related to the transition, and selecting a number of tiles with the highest confidence to calibrate the prediction results.

9. The method according to claim 1, characterized in that, The model's loss function is: in, p is a hyperparameter, q is a hyperparameter balancing noise immunity and convergence, and p is a hyperparameter. y It is the predicted probability of label y. For slice-level visual-language alignment contrast loss, The optimization objective is for tile-level pseudo-labels.

10. A computing device, characterized in that, The device includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the computing device to perform the method according to any one of claims 1 to 8.

Images