A multi-modal survival prediction method based on multi-dimension score and confidence perception

Abstract

This invention provides a multimodal survival prediction method based on multidimensional scoring and confidence perception. The method includes a comprehensive pathology-gene pair association score, normal-inverse gamma distribution modeling, and confidence-weighted fusion. The comprehensive pathology-gene pair association score achieves precise alignment between pathological image regions and functional gene modules through comprehensive semantic similarity, functional attribution gradient, and structural matching information. In the normal-inverse gamma distribution modeling part, each modality is modeled separately using the normal-inverse gamma distribution, explicitly quantifying the uncertainty in the prediction. In the confidence-weighted fusion, a confidence-weighted fusion strategy based on the Student-t distribution is proposed, dynamically adjusting the fusion weights according to the uncertainty of each modality's prediction, and ensuring that the fusion result does not reduce the overall confidence through a confidence ranking regularization term. This invention solves the problems of difficult multimodal semantic alignment, insufficient confidence, and rigid fusion strategies in existing multimodal survival prediction methods, significantly improving the accuracy and stability of survival prediction.

Description

Technical Field

[0001] This invention relates to the field of pathological image and gene data processing technology, and in particular to a multimodal survival prediction method based on multidimensional scoring and confidence perception. Background Technology

[0002] Multimodal survival prediction integrating histopathological images and genomic data is a key technology for improving precision cancer diagnosis and treatment. However, traditional methods in the current field of cancer survival prediction have many shortcomings. First, there is a semantic disconnect between the morphological features of histopathological images and the molecular features of genomic data, making it difficult for traditional methods to achieve accurate cross-modal alignment. Second, traditional methods only output a single risk value, failing to consider the uncertainty brought about by data noise, resulting in insufficient reliability of the prediction results. Third, traditional methods use fixed weights to fuse multimodal features, ignoring the reliability differences of each modality in different samples; this rigid fusion method reduces the robustness of the model. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a multi-modal survival prediction method based on multi-dimensional scoring and confidence awareness (MSCA). By employing gene-pathology comprehensive association scoring, normal-inverse gamma (NIG) distribution modeling, and Student-t confidence weighted fusion, this method solves the problems of difficult multi-modal semantic alignment, insufficient confidence, and rigid fusion in existing technologies, thereby improving the accuracy and clinical applicability of cancer survival prediction.

[0004] This invention proposes a multimodal survival prediction method based on multidimensional scoring and confidence perception, comprising the following steps:

[0005] Step S1: Preprocess the collected multimodal data;

[0006] Step S2, Pathology-Gene Pair Comprehensive Association Score: Given a set of pathology image patch embeddings and embedding of genome functional modules By integrating semantic similarity scoring, functional attribution scoring, and structural matching scoring through the multi-dimensional comprehensive association scoring (MCAS) mechanism, alignment of image regions with gene modules can be achieved.

[0007] Step S3: Normal-inverse gamma NIG distribution modeling. After NIG joint modeling, the Student-t distribution is obtained, and the patient's risk prediction value is output.

[0008] Step S4, confidence-weighted fusion, fuses the outputs of gene modalities and pathological modalities to obtain the fused output distribution;

[0009] Step S5, Total Loss Analysis: The partial likelihood of the Cox proportional hazards model is used as the ranking supervision signal to optimize the consistency between the predicted mean after fusion and the actual survival ranking, and to construct the survival loss. Then, the uncertainty of the output of the pathological image modality and the gene modality is modeled by the normal-inverse gamma distribution NIG to obtain the NIG modeling loss. For the supervision after fusion, the confidence-weighted fusion part loss is calculated by using the Student-t distribution parameters obtained by fusion.

[0010] Further, step S1 specifically includes:

[0011] For whole-slice images of pathology, WSI is denoted as... Based on multi-instance learning, each WSI is represented as containing The package of permutation-invariant instances , Indicates the first An example, denoted using a ResNet-50 encoder, is... deal with To obtain pathological image feature packages , is represented as: ,in, Indicates the first Features of an image instance;

[0012] For genomic data, it is divided into categories based on biological function. A class, representing an expressive genome package as ,in Indicates the first 1 genome, denoted by an SNN encoder as deal with Obtain gene data feature package , is represented as: ,in, Indicates the first Characteristics of a genome instance.

[0013] Furthermore, in step S2, semantic similarity scoring... The calculation method is as follows:

[0014] ,

[0015] in, Indicates the first Features of an image instance Indicates the first Characteristics of a genome instance;

[0016] The causal effect of gene expression on pathological features is calculated by gradient backpropagation, thus obtaining the functional attribution score. The calculation method is as follows:

[0017] ,

[0018] in, For pathological feature encoder, Indicates the first Features of an image instance Indicates the first Characteristics of a genome instance This indicates the partial derivative;

[0019] The structural matching score is obtained by using alignment probabilities based on radial basis function kernels. The calculation method is as follows:

[0020] ,

[0021] in, Represents the square of the L2 norm. Indicates the first Features of an image instance Indicates the first Characteristics of a genome instance;

[0022] By integrating the scores from the three dimensions mentioned above, a comprehensive correlation score for the gene-pathology pair is obtained. : ,in , , These are the weighting coefficients.

[0023] Furthermore, step S3 specifically includes:

[0024] Using the Transformer model, denoted as Predict the risk score for a specific survival time point from each instance of a single modality, denoted as ,in, , The predicted risk score of an image instance is represented by... , The predicted risk score of a genome instance is represented by... ;

[0025] Set risk score Follows Gaussian distribution The mean of this Gaussian distribution and variance Governed by the prior evidence of the NIG distribution, the parameters of the NIG distribution are expressed as follows: ,in, Represents the predicted mean in a Gaussian distribution Prior expectations, It reflects Uncertainty in precision parameters It is the shape parameter of the NIG distribution. The scaling parameter of the NIG distribution is the marginal distribution obtained after joint modeling by NIG, which is the Student-t distribution, and its mean parameter is... = ,variance The calculation method is as follows: Degrees of freedom parameters The calculation method is as follows: Prediction results That is, the distribution of the patient's expected survival probability is as follows: .

[0026] Furthermore, step S4 specifically includes:

[0027] The output distribution of gene modalities is defined as: ;

[0028] The pathological modality output distribution is defined as: ;

[0029] Assumption > The gene modality and pathological modality are fused using inverse variance weighting, resulting in the following fused output distribution:

[0030] ,

[0031] Among them, the fusion prediction mean The calculation method is as follows: , This represents the mean of the Student-t distribution corresponding to the pathological modality. This represents the mean of the Student-t distribution corresponding to the gene modality. and The calculation method is as follows:

[0032] ,

[0033] The calculation method is as follows: ,in, This represents the variance of the Student-t distribution corresponding to the pathological modality. This represents the variance of the Student-t distribution corresponding to the gene modality;

[0034] Fusion of degrees of freedom parameters ;

[0035] To ensure that the confidence level does not decrease after fusion, a ranking regularization term is introduced. :

[0036] ,

[0037] in, , and Let represent the variances of the image modality, gene modality, and the Student-t distribution corresponding to the fusion, respectively.

[0038] Further, the survival loss in step S5 is defined as:

[0039] ,

[0040] in, Indicates the first The mean of the multimodal fusion prediction for each sample. Indicates the first Risk set of a sample Indicates the relationship with the first A set of gene features associated with a pathological sample. Indicating pathological samples and gene samples The overall correlation score.

[0041] Furthermore, the NIG modeling loss in step S5 is:

[0042] ,

[0043] in, Let NIG be the prior expected parameter in the NIG distribution. It is a precision parameter. These are shape parameters. It is a scale parameter. For actual survival time, Represents the gamma function. ;

[0044] The sum of the NIG modeling losses for pathological image modalities and gene modalities is expressed as:

[0045] ,

[0046] in, This represents the NIG modeling loss of pathological image modalities. This indicates the loss in NIG modeling of gene modalities. This represents the NIG modeling loss after addition.

[0047] Furthermore, in step S5, the supervision after fusion is performed using the Student-t distribution parameters obtained through fusion. Calculate its negative log-likelihood with respect to the true target value. This loss term is defined as follows:

[0048] ,

[0049] in, , , These represent the mean, variance, and degrees of freedom parameters of the merged Student-t distribution, respectively.

[0050] Introducing a confidence ranking regularization term to constrain the fusion variance to be no higher than the single-modal variance:

[0051] ;

[0052] The loss of the confidence-weighted fusion component is:

[0053] ;

[0054] The total loss function of the multimodal survival prediction method based on multidimensional scoring and confidence perception is:

[0055] ;

[0056] in, , , This is a hyperparameter.

[0057] This invention proposes a multimodal survival prediction method based on multidimensional scoring and confidence perception. The method mainly consists of three parts: pathology-gene pair comprehensive association scoring, normal-inverse gamma distribution modeling, and confidence-weighted fusion. The pathology-gene pair comprehensive association scoring achieves accurate alignment between pathological image regions and functional gene modules by integrating semantic similarity, functional attribution gradient, and structural matching information. In the normal-inverse gamma distribution modeling part, each modality is modeled separately using the normal-inverse gamma distribution to explicitly quantify the uncertainty in the prediction. In the confidence-weighted fusion, a confidence-weighted fusion strategy based on the Student-t distribution is proposed. The fusion weights are dynamically adjusted according to the uncertainty of each modality prediction, and a confidence ranking regularization term is used to ensure that the fusion result does not reduce the overall confidence. This invention solves the problems of difficult multimodal semantic alignment, insufficient confidence, and rigid fusion strategies in existing multimodal survival prediction methods, significantly improving the accuracy and stability of survival prediction. It realizes adaptive alignment and confidence-perceived fusion of cross-modal features, providing more reliable technical support for clinical personalized prognostic assessment and decision support. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart illustrating a multimodal survival prediction method based on multidimensional scoring and confidence perception provided in an embodiment of the present invention.

[0060] Figure 2 This is a schematic diagram illustrating the principle of confidence-weighted fusion provided in an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] This invention proposes a multimodal survival prediction method based on multidimensional scoring and confidence perception, such as... Figure 1 As shown, the method includes the following steps:

[0063] Step S1: Multimodal data preprocessing. For whole slide images (WSI), based on multi-instance learning, each WSI (denoted as...) is preprocessed... ) indicates inclusion The package of permutation-invariant instances , To represent a specific instance, use a ResNet-50 encoder (denoted as...). )deal with To obtain pathological image feature packages , is represented as: ,in, Indicates the first Features of an image instance;

[0064] For genomic data, it is divided into categories based on biological function. A class, representing an expressive genome package as ,in To represent a genome, use an SNN encoder (denoted as ). )deal with To obtain gene data feature packages , is represented as: ,in, Indicates the first Characteristics of a genome instance.

[0065] Step S2: Comprehensive association scoring of pathology-gene pairs. Given a set of pathology image patch embeddings. and embedding of genome functional modules By integrating semantic similarity scoring, functional attribution scoring, and structural matching scoring through a multi-dimensional comprehensive association scoring mechanism (MCAS), precise alignment of image regions with gene modules is achieved. Semantic similarity scoring... The calculation method is as follows:

[0066] ,

[0067] in, Indicates the first Features of an image instance Indicates the first Features of each genome instance. This score measures the direct similarity between two modalities in the feature space, highlighting the local correspondence between epipathological morphology and molecular function. The causal influence of gene expression on pathological features is calculated through gradient backpropagation, thus obtaining the functional attribution score. The calculation method is as follows:

[0068] ,

[0069] in, For pathological feature encoder, Indicates the first Features of an image instance Indicates the first Characteristics of a genome instance This represents the partial derivative. This scoring quantification gene module... For image patches The contribution of features reveals potential biological regulatory relationships.

[0070] Structural matching focuses on the local geometric alignment between pathological image instances and genomic instances, capturing modal relationships that may not be apparent in semantic space. This invention uses alignment probabilities based on radial basis function kernels to obtain a structural matching score. The calculation method is as follows:

[0071] ,

[0072] in, Represents the square of the L2 norm. Indicates the first Features of an image instance Indicates the first Features of each genome instance and a structural matching score are used to ensure structural consistency between modalities. By fusing the above three-dimensional scores, a comprehensive association score for gene-pathology pairs is obtained. :

[0073] ,

[0074] in , , , which is a weighting coefficient used to balance the contributions of semantic, functional, and structural information.

[0075] Step S3: Normal-Inverse Gamma (NIG) Distribution Modeling. In survival prediction analysis, the model needs to output a risk prediction value for each patient, but this alone is insufficient, especially in scenarios where a certain modality is missing. Therefore, this invention aims to provide not only the prediction value but also the confidence level of that prediction value. To this end, this invention employs a Bayesian regression modeling framework: Normal-Inverse Gamma (NIG) distribution modeling. Specifically, a Transformer (denoted as...) is first used... The model predicts the risk score at a specific survival time point from each instance of a single modality, denoted as . ,in . The predicted risk score of an image instance is represented by... ,and The predicted risk score of a genome instance is represented by... Assuming a risk score Follows Gaussian distribution The mean of this Gaussian distribution and variance Governed by the prior evidence of the NIG distribution, the parameters of the NIG distribution are expressed as follows: ,in Represents the predicted mean in a Gaussian distribution Prior expectations, It reflects Uncertainty in precision parameters It is the shape parameter of the NIG distribution. It is its scale parameter. The marginal distribution obtained after joint modeling by NIG is a Student-t distribution, and its mean parameter is... = ,variance The calculation method is as follows: ;

[0076] Degrees of freedom parameters The calculation method is as follows: ;

[0077] Prediction results That is, the distribution of the patient's expected survival probability is as follows: ;

[0078] This modeling approach enhances robustness to data heterogeneity and censored labels.

[0079] Step S4: Confidence-weighted fusion. The principle of confidence-weighted fusion is as follows: Figure 2 As shown, the output distribution of gene modalities is defined as follows: ;

[0080] The pathological modality output distribution is defined as: ;

[0081] Assumption > Genetic and pathological modalities are fused using inverse variance weighting, and the fused output distribution is defined as follows: ;

[0082] Among them, the fusion prediction mean The calculation method is as follows: , This represents the mean of the Student-t distribution corresponding to the pathological modality. This represents the mean of the Student-t distribution corresponding to the gene modality. and The calculation method is as follows:

[0083] ,

[0084] The calculation method is as follows: ,in, This represents the variance of the Student-t distribution corresponding to the pathological modality. This represents the variance of the Student-t distribution corresponding to the gene modality;

[0085] Fusion of degrees of freedom parameters .

[0086] To ensure that the confidence level does not decrease after fusion, a ranking regularization term is introduced. : ,in, , and Let represent the variances of the image modality, gene modality, and the Student-t distribution corresponding to the fusion, respectively.

[0087] Step S5: Total Loss Analysis. First, the partial likelihood of the Cox proportional hazards model is used as the primary ranking supervision signal to optimize the fused prediction mean. Consistency with the actual survival ranking, let the first... The deletion indication for each sample is: , Indicates the first The samples are uncensored, and their risk set is: The risk set is obtained from the raw data, and the survival loss is defined as:

[0088] ,

[0089] in, Indicates the first The mean of the multimodal fusion prediction for each sample. Indicates the first Risk set of a sample Indicates the relationship with the first A set of gene features associated with a pathological sample. Indicating pathological samples and gene samples The overall correlation score.

[0090] Secondly, a normal-inverse gamma (NIG) distribution is used for each mode. The output is used to model uncertainty, and its negative logarithmic form of marginal log-likelihood is used as the supervision loss, i.e., the NIG modeling loss:

[0091] ,

[0092] in, Let NIG be the prior expected parameter in the NIG distribution. It is a precision parameter. These are shape parameters. It is a scale parameter. For actual survival time, Let gamma function be the sum of the NIG modeling losses for pathological image modalities and gene modalities. ,in, This represents the NIG modeling loss of pathological image modalities. This indicates the loss in NIG modeling of gene modalities. This represents the NIG modeling loss after addition.

[0093] For the supervision after fusion, the Student-t distribution parameters obtained by fusion are used. Calculate its negative log-likelihood with respect to the true target value. This loss term is defined as follows:

[0094] ,

[0095] in, , , Let represent the mean, variance, and degrees of freedom parameters of the fused Student-t distribution, respectively. To ensure that the confidence level of the fused result is superior to any single modality, this invention introduces a confidence ranking regularization term to constrain the fused variance to be no higher than the single-modality variance:

[0096] ;

[0097] Therefore, the loss of the confidence-weighted fusion component is:

[0098] ;

[0099] In summary, the total loss function of the multimodal survival prediction method based on multidimensional scoring and confidence perception is:

[0100] ;

[0101] in, , , This is a hyperparameter used to balance the weights of the three components.

[0102] Based on the analysis of the experimental results, , , The model performs best when the values ​​are 0.2, 0.4, and 0.4, therefore, in this embodiment... , , The preferred values ​​are 0.2, 0.4, and 0.4.

[0103] The method of the present invention will be experimentally verified using specific data below:

[0104] To validate the effectiveness of the method, experiments were conducted on two publicly available cancer datasets from The Cancer Genome Atlas (TCGA): the Lung Adenocarcinoma (LUAD) dataset and the Bladder Urothelial Carcinoma (BLCA) dataset. These datasets contain paired whole-section pathological images and genomic data with labeled survival results. The LUAD dataset contains 453 cases, and the BLCA dataset contains 373 cases.

[0105] This embodiment of the experiment is implemented using the PyTorch deep learning framework with an NVIDIA A100 GPU, employing the Adam optimizer, and setting the initial learning rate to 2×10⁻⁻⁻⁻⁶. 4 The weight decay coefficient is 1×10⁻ 5 Considering the large size of a single pathological image, the batch size was set to 1 during training, and a total of 50 rounds of iterative training were completed. The micro-batch size was set to 256. Experimental evaluation used 5-fold cross-validation, dividing the training and validation sets in a 4:1 ratio. The cross-validation consistency index (C-Index) and the time-dependent area under the curve (tAUC) were used as performance metrics. These two metrics quantify the model's accuracy in ranking patient risk scores and overall survival rates; higher values ​​indicate better model performance.

[0106] Table 1 shows the test results of the MSCA method of this invention and other survival prediction algorithms on the LUAD and BLCA datasets. As can be seen from the table, the method of this invention outperforms other methods in both C-Index and tAUC metrics. The test results of other methods are not ideal, mainly due to existing methods' problems of difficult multimodal semantic alignment, insufficient reliability, and rigid fusion. In contrast, the method of this invention achieves excellent results, with all metrics significantly superior to other methods. This reflects that this invention can effectively solve the existing problems of difficult multimodal semantic alignment, insufficient reliability, and rigid fusion.

[0107] Table 1. Comparison of evaluation metrics between the method of this invention and other methods on the LUAD and LUSC datasets.

[0108]

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal survival prediction method based on multidimensional scoring and confidence perception, characterized in that, The method includes: Step S1: Preprocess the collected multimodal data; Step S2, Pathology-Gene Pair Comprehensive Association Score: Given a set of pathology image patch embeddings and embedding of genome functional modules By integrating semantic similarity scoring, functional attribution scoring, and structural matching scoring through the multi-dimensional comprehensive association scoring (MCAS) mechanism, alignment of image regions with gene modules can be achieved. Step S3: Normal-inverse gamma NIG distribution modeling. After NIG joint modeling, the Student-t distribution is obtained, and the patient's risk prediction value is output. Step S4, confidence-weighted fusion, fuses the outputs of gene modalities and pathological modalities to obtain the fused output distribution; Step S5, Total Loss Analysis: The partial likelihood of the Cox proportional hazards model is used as the ranking supervision signal to optimize the consistency between the predicted mean after fusion and the actual survival ranking, and to construct the survival loss. Then, the uncertainty of the output of the pathological image modality and the gene modality is modeled by the normal-inverse gamma distribution NIG to obtain the NIG modeling loss. For the supervision after fusion, the confidence-weighted fusion part loss is calculated by using the Student-t distribution parameters obtained by fusion.

2. The multimodal survival prediction method based on multidimensional scoring and confidence perception according to claim 1, characterized in that, Step S1 further includes: For whole-slice images of pathology, WSI is denoted as... Based on multi-instance learning, each WSI is represented as containing The package of permutation-invariant instances , Indicates the first An example, denoted using a ResNet-50 encoder, is... deal with To obtain pathological image feature packages , is represented as: ,in, Indicates the first Features of an image instance; For genomic data, it is divided into categories based on biological function. A class, representing an expressive genome package as ,in Indicates the first 1 genome, denoted by an SNN encoder as deal with Obtain gene data feature package , is represented as: ,in, Indicates the first Characteristics of a genome instance.

3. The multimodal survival prediction method based on multidimensional scoring and confidence perception according to claim 2, characterized in that, Semantic similarity scoring in step S2 The calculation method is as follows: ， in, Indicates the first Features of an image instance Indicates the first Characteristics of a genome instance; The causal effect of gene expression on pathological features is calculated by gradient backpropagation, thus obtaining the functional attribution score. The calculation method is as follows: ， in, For pathological feature encoder, Indicates the first Features of an image instance Indicates the first Characteristics of a genome instance This indicates the partial derivative; The structural matching score is obtained by using alignment probabilities based on radial basis function kernels. The calculation method is as follows: ， in, Represents the square of the L2 norm. Indicates the first Features of an image instance Indicates the first Characteristics of a genome instance; By integrating the scores from the three dimensions mentioned above, a comprehensive correlation score for the gene-pathology pair is obtained. : ,in , , These are the weighting coefficients.

4. The multimodal survival prediction method based on multidimensional scoring and confidence perception according to claim 3, characterized in that, Step S3 further includes: Using the Transformer model, denoted as Predict the risk score for a specific survival time point from each instance of a single modality, denoted as ,in, , The predicted risk score of an image instance is represented by... , The predicted risk score of a genome instance is represented by... ; Set risk score Follows Gaussian distribution The mean of this Gaussian distribution and variance Governed by the prior evidence of the NIG distribution, the parameters of the NIG distribution are expressed as follows: ,in, Represents the predicted mean in a Gaussian distribution Prior expectations, It reflects Uncertainty in precision parameters It is the shape parameter of the NIG distribution. The scaling parameter of the NIG distribution is the marginal distribution obtained after joint modeling by NIG, which is the Student-t distribution, and its mean parameter is... = ,variance The calculation method is as follows: Degrees of freedom parameters The calculation method is as follows: Prediction results That is, the distribution of the patient's expected survival probability is as follows: .

5. The multimodal survival prediction method based on multidimensional scoring and confidence perception according to claim 4, characterized in that, Step S4 further includes: The output distribution of gene modalities is defined as: ; The pathological modality output distribution is defined as: ; Assumption > The gene modality and pathological modality are fused using inverse variance weighting, resulting in the following fused output distribution: ， Among them, the fusion prediction mean The calculation method is as follows: , This represents the mean of the Student-t distribution corresponding to the pathological modality. This represents the mean of the Student-t distribution corresponding to the gene modality. and The calculation method is as follows: ， The calculation method is as follows: ,in, This represents the variance of the Student-t distribution corresponding to the pathological modality. This represents the variance of the Student-t distribution corresponding to the gene modality; Fusion of degrees of freedom parameters ; To ensure that the confidence level does not decrease after fusion, a ranking regularization term is introduced. : ， in, , and Let Variance represent the variances of the image modality, gene modality, and the Student-t distribution corresponding to the fusion, respectively.

6. The multimodal survival prediction method based on multidimensional scoring and confidence perception according to claim 5, characterized in that, In step S5, the survival loss is defined as: ， in, Indicates the first The mean of the multimodal fusion prediction for each sample. Indicates the first Risk set of a sample Indicates the relationship with the first A set of gene features associated with a pathological sample. Indicating pathological samples and gene samples The overall correlation score.

7. The multimodal survival prediction method based on multidimensional scoring and confidence perception according to claim 6, characterized in that, The NIG modeling loss in step S5 is: ， in, Let NIG be the prior expected parameter in the NIG distribution. It is a precision parameter. These are shape parameters. It is a scale parameter. For actual survival time, Represents the gamma function. ; The sum of the NIG modeling losses for pathological image modalities and gene modalities is expressed as: ， in, This represents the NIG modeling loss of pathological image modalities. This indicates the loss in NIG modeling of gene modalities. This represents the NIG modeling loss after addition.

8. The multimodal survival prediction method based on multidimensional scoring and confidence perception according to claim 7, characterized in that, In step S5, the supervision after fusion is performed using the Student-t distribution parameters obtained through fusion. Calculate its negative log-likelihood with respect to the true target value. This loss term is defined as follows: ， in, , , These represent the mean, variance, and degrees of freedom parameters of the merged Student-t distribution, respectively. Introducing a confidence ranking regularization term to constrain the fusion variance to be no higher than the single-modal variance: ； The loss of the confidence-weighted fusion component is: ； The total loss function of the multimodal survival prediction method based on multidimensional scoring and confidence perception is: ； in, , , This is a hyperparameter.

Images