A multi-modal survival prediction method based on information bottleneck and dynamic weighted fusion

Abstract

The application provides a multi-modal survival prediction method based on information bottleneck and dynamic weighted fusion, which comprises intra-modal information purification and inter-modal dynamic fusion. In the intra-modal information purification stage, an information bottleneck mechanism is introduced to minimize the correlation between the input and the feature layer, suppress background noise, and maximize the correlation between the feature layer and the survival label, so as to retain the key features most relevant to the survival outcome and realize efficient information compression of the pathological and genetic modalities. In the inter-modal fusion stage, a dynamic weighted fusion strategy is proposed to adjust the fusion weight according to the prediction uncertainty of each modality, so that the model can automatically reduce the influence of a certain modality when the noise of the modality is strong or the data quality is uneven, and enhance the overall robustness. The application scheme combines pathological whole slide images and genomic data, realizes structured feature learning through a graph neural network, and significantly improves the accuracy, stability and interpretability of cancer survival prediction under the joint action of information bottleneck constraint and dynamic weight scheduling, providing reliable intelligent support for clinical precise prognosis evaluation.

Description

Technical Field

[0001] This invention relates to the field of pathological image and genomic data processing technology, and in particular to a multimodal survival prediction method based on information bottleneck and dynamic weighted fusion. Background Technology

[0002] Multimodal survival prediction that integrates whole-slice pathological images with genomic data is a core technology for improving precision diagnosis and prognostic assessment of cancer. Current methods have two major drawbacks: First, most existing methods use simple feature splicing or static weight fusion strategies, without considering the differences in information content between different modalities, resulting in weak modalities dragging down the overall performance of the model; Second, noise and useful information are highly coupled in weak modalities, and existing methods have difficulty in effectively removing noise, thus interfering with the model's extraction of effective features. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a multimodal survival prediction method based on information bottleneck and dynamic weighted fusion. It constructs a two-stage strategy of "purification first, then scheduling," that is, information compression within a modality is achieved through an information bottleneck mechanism, and adaptive scheduling between modalities is achieved through dynamic weighted fusion. This solves the problems of information imbalance and noise interference in existing technologies, and significantly improves the accuracy and robustness of cancer survival prediction.

[0004] A multimodal survival prediction method based on information bottleneck and dynamic weighted fusion includes the following steps:

[0005] Step S1, Pathological image modeling: Define the pathological image as a directed graph, and use a graph neural network to learn graph features to obtain pathological representations;

[0006] Step S2, Genomic Data Modeling: Screen differentially expressed genes, analyze the pairwise correlation of the screened genes, binarize the adjacency matrix to generate the edge matrix, and finally input the genomic graph network into the graph attention network layer to construct a multi-layer graph representation;

[0007] Step S3, Survival prediction task modeling: Format patient data and define survival prediction risk function and survival function;

[0008] Step S4, Intramodal Information Compression Based on Information Bottleneck: An information bottleneck mechanism is introduced within the modality to refine the representation of each graph;

[0009] Step S5, Dynamic weighted fusion of multimodal information: Adaptively adjust the weight of each modality based on the uncertainty of each sample to the prediction result;

[0010] Step S6, Survival prediction loss: Add the supervision loss to the total information bottleneck loss to obtain the total survival prediction loss.

[0011] Further, step S1 specifically includes:

[0012] The pathological images were cropped into a set of 256×256 image blocks. ,in, For image blocks, Given the total number of image patches, a 512-dimensional representation of each image patch is extracted using the pre-trained model PLIP, resulting in... ,in, For eigenvalues;

[0013] Each image patch is then defined as a directed graph. , where nodes Corresponding image patch representation, edge The connectivity is dynamically updated during training. The graph features are learned using a sampling-aggregation approach with GraphSAGE graph neural networks. For the first ( Nodes in layer The representation, each layer first through Nearest neighbor node found direct neighborhood , For neighborhood set, Represent any neighboring node in the neighborhood set, and then use the aggregation function. Aggregate neighborhood representations into Then represent the current node. and The concatenated input includes a non-linear activation function. The fully connected layer is updated in the following way:

[0014]

[0015] in, For the first Layer learnable parameters For splicing operations; The design employs residual mapping and implements dense connections from the output of each layer to the last layer. Finally, it adaptively calculates the weighted sum of features of all nodes to obtain the pathological characterization.

[0016] Further, step S2 specifically includes:

[0017] Before screening from the patient cohort Identify differentially expressed genes associated with prognosis, and then use weighted gene co-expression network analysis to calculate the results after screening. The pairwise correlations of each gene are used to obtain the adjacency matrix. Then, by adjusting the adjacency matrix Binarization generates the edge matrix. Finally, the genome map network A three-layer graph attention network (GAT) is used to construct a multi-layer graph representation. The node features of these multi-layer representations are then projected and concatenated to obtain a genome representation that simultaneously preserves individual gene expression information and inter-gene co-expression relationships. The genome graph network... nodes For a single patient Gene expression values, with sides as .

[0018] Furthermore, step S3 specifically includes:

[0019] Each patient's data is represented as a quadruple. ,in, and These represent the pathological images processed in step S1 and the genomic data processed in step S2, respectively. This is a right censoring state, indicating whether the resulting event was observed. Indicates that it was observed, This indicates that it was not observed. This is the total survival time, divided into n intervals, each interval represented as... Survival prediction risk function Defined as an interval The probability of failure within, The expression is as follows:

[0020]

[0021] in, This represents the random time at which a patient's survival outcome event occurs. The learnable parameters of the survival model are represented. Indicates pathology and genome Patient survival-related joint feature representations learned from data;

[0022] Survival function The sequence risk used to estimate the outcome event is obtained by taking the negative of the cumulative risk function, and is expressed as:

[0023]

[0024] in, For survival function, Represents a variable.

[0025] Furthermore, step S4 specifically includes:

[0026] The information bottleneck mechanism is represented as follows:

[0027]

[0028] in, The original input data, This represents the feature representation of the intermediate layer of a neural network. The label value is the patient's survival time. The balancing coefficient, dCor, represents the distance correlation coefficient, which is defined as:

[0029]

[0030] in, It is to prevent extremely small positive numbers with a denominator of zero. express and The distance covariance between them measures the degree of dependence between them, and is defined as:

[0031]

[0032] Where N represents the number of samples, and These represent the input variables respectively. and intermediate features The elements of the sample distance matrix after doubly centering and The definitions of are respectively:

[0033]

[0034] Where N represents the number of samples, and They represent and The elements of the sample distance matrix after doubly centering;

[0035] The total information bottleneck loss for both modes is:

[0036]

[0037] in, This represents the total information bottleneck loss. and These represent the information bottleneck losses of the image modality and the genome modality, respectively. and These represent the intermediate layers of the neural network during training for the image modality and the genome modality, respectively.

[0038] Further, step S5 specifically includes:

[0039] First, the pathological images and genomic features after processing in steps S1 and S2 are defined as follows: and The uncertainties of pathological and gene modalities are extracted separately and expressed as follows:

[0040]

[0041] in, and These represent the uncertainty parameters for pathological features and genetic features, respectively. It is an activation function. and These are uncertainty estimation networks for pathological and genetic modalities, respectively.

[0042] Then, the pathological and genomic features are input into the cross-modal joint expert, and the uncertainty of the cross-modal joint features is calculated, expressed as:

[0043]

[0044] in, Indicates cross-modal joint experts, This represents the combined characteristics of pathological and genomic modalities. This indicates the uncertainty of cross-modal joint features. An uncertainty estimation network representing cross-modal joint features;

[0045] Then, the modal features and their uncertainties are input into the hypernetwork to dynamically generate the routing parameters for the current patient:

[0046]

[0047] in, This indicates a feature concatenation operation. Indicates a dynamic routing network. This refers to routing parameters dynamically generated based on the current patient's modal characteristics and their uncertainties;

[0048] The routing scores for pathologists, genomics experts, and cross-modal joint experts are calculated based on dynamic routing parameters:

[0049]

[0050] in, , , Let represent the routing scores of pathology experts, genomics experts, and cross-modal joint experts, respectively. Considering the uncertainty associated with each expert, a penalty is applied to the routing scores, and a dynamic fusion weight is obtained using the Softmax function.

[0051]

[0052] in, , , These represent the dynamic fusion weights of pathologists, genomics experts, and cross-modal collaborative experts, respectively. This represents the uncertainty penalty coefficient. Finally, the three expert features are weighted and summed according to the dynamic fusion weights to obtain the final fused feature:

[0053]

[0054] in, As a pathology expert, As a genetics expert, For cross-modal joint experts.

[0055] Furthermore, the supervision loss in step S6 is expressed as:

[0056]

[0057] in, and Let these represent the risk function and the survival function, respectively.

[0058] The total loss is: .

[0059] This invention proposes a multimodal survival prediction method based on information bottleneck and dynamic weighted fusion. The method mainly includes two core stages: intramodal information purification and intermodal dynamic fusion. In the intramodal information purification stage, an information bottleneck mechanism is introduced to minimize the correlation between the input and the feature layer, suppress background noise, and maximize the correlation between the feature layer and the survival label, thereby retaining the key features most relevant to the survival outcome and achieving efficient information compression of both pathological and genetic modalities. In the intermodal fusion stage, a dynamic weighted fusion strategy is proposed, which adaptively adjusts the fusion weights of each modality according to the uncertainty of the prediction results. This allows the model to automatically reduce the impact of a particular modality when noise is strong or the data quality is uneven, thus enhancing overall robustness. This method combines pathological whole-slice images and genomic data, and achieves structured feature learning through graph neural networks. Under the combined effect of information bottleneck constraints and dynamic weight scheduling, it effectively alleviates the problems of information imbalance, noise coupling, and fusion rigidity in traditional multimodal models, significantly improving the accuracy, stability, and interpretability of cancer survival prediction, and providing reliable intelligent support for precise clinical prognostic assessment. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart illustrating a multimodal survival prediction method based on information bottleneck and dynamic weighted fusion provided in an embodiment of the present invention.

[0062] Figure 2 This is a schematic diagram illustrating the principle of dynamic weighted fusion provided in an embodiment of the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] This invention proposes a multimodal cancer survival prediction method based on information bottleneck and dynamic weighted fusion, such as... Figure 1 As shown, the method includes the following steps:

[0065] Step S1: Pathological image modeling. First, the pathological image is cropped into a set of 256×256 image patches at a magnification of 20x. ,in, Image blocks, i.e., the raw image data. Given the total number of image patches, a 512-dimensional representation of each image patch is extracted using the pre-trained model PLIP, resulting in... ,in, These are the feature values. Each image patch is then defined as a directed graph. , where nodes Corresponding image patch representation, edge The connectivity is dynamically updated during training. The graph features are learned using a sampling-aggregation approach with GraphSAGE graph neural networks. For the first ( Nodes in layer ( For the representation of nodes in the GraphSAGE graph neural network, each layer first passes through... Nearest neighbor node found direct neighborhood , For neighborhood set, Represent any neighboring node in the neighborhood set, and then use the aggregation function. Aggregate neighborhood representations into Then represent the current node. and The concatenated input includes a non-linear activation function. The fully connected layer is updated in the following way:

[0066]

[0067] in, For the first Layer learnable parameters For splicing operations; for comprehensively capturing features, The design employs residual mapping (adding the output to the input) and implements dense connections from the output of each layer to the last layer, enabling image patch representations to integrate their own information with the surrounding microenvironment. Finally, the weighted sum of all node features is adaptively calculated to obtain the pathological representation.

[0068] Step S2: Genomic Data Modeling. First, differential expression analysis (DEA) is used to screen patients from the cohort based on the false discovery rate (FDR < 0.05) and log fold change. Among the differentially expressed genes associated with prognosis, those with significant expression changes and biological significance were prioritized to improve predictive interpretability; then, weighted gene co-expression network analysis was used to calculate the results after screening. The pairwise correlations of each gene are used to obtain the adjacency matrix. (The higher the matrix value, the more similar the expression patterns of the two corresponding genes), and then by analyzing the adjacency matrix... Binarization generates the edge matrix. Furthermore, this edge matrix is ​​calculated based on all patient data, is only related to gene characteristics, and does not need to be repeatedly generated for individual patients. Finally, the genome graph network... (node For a single patient Gene expression values, with sides as The input is a 3-layer Graph Attention Network (GAT) layer to construct a multi-layer graph representation. The node features of the multi-layer representation are projected and spliced ​​to obtain a genome representation that retains both the expression information of individual genes and the co-expression relationship between genes.

[0069] Step S3: Survival prediction task modeling. Survival prediction aims to estimate the probability of a death event occurring within a specific time period. In multimodal survival prediction, the goal of this invention is to develop a survival model. The model uses pathological image data and genomic data to estimate the risk function. Given a containing A dataset of patients, where each patient's data is represented as a quaternion. ,in, and These represent the pathological images processed in step S1 and the genomic data processed in step S2, respectively. It is a right-censored state, indicating whether the resulting event was observed. Indicates that it was observed, (Indicates not observed) It is the total survival time. Let be the set of positive real numbers. In practical applications, instead of directly predicting the exact time of patient outcome events, survival time is divided into n intervals, each interval represented as . And approximate the result time within the discretized interval. Survival prediction risk function Defined as an interval The probability of failure within, The expression is as follows:

[0070]

[0071] in, This represents the random time at which a patient's survival outcome event occurs. The learnable parameters of the survival model are represented. Indicates pathology and genome Patient survival-related joint feature representations learned from data;

[0072] Survival function The sequence risk used to estimate the outcome event is obtained by taking the negative of the cumulative risk function, and is expressed as:

[0073]

[0074] in, For survival function, Represents a variable.

[0075] Step S4: Intramodal information compression based on information bottleneck. Within a modality, an information bottleneck mechanism is introduced to refine the representation of each graph, minimizing the correlation between the input and the feature layer to suppress background noise, while maximizing the correlation between the feature layer and the label to retain key survival-related information. The information bottleneck mechanism is generally expressed as follows:

[0076]

[0077] in, The original input data, This represents the feature representation of the intermediate layer of a neural network. The label value is the patient's survival time. The balancing coefficient, dCor, represents the distance correlation coefficient, which is defined as:

[0078]

[0079] in, It is to prevent extremely small positive numbers with a denominator of zero. express and The distance covariance between them measures the degree of dependence between them, and is defined as:

[0080]

[0081] Where N represents the number of samples, and These represent the input variables respectively. and intermediate features The elements of the sample distance matrix after double centering are: The sample distance matrix is ​​a matrix formed by calculating the distance between any two samples, also called the Pairwise Euclidean Distance Matrix. Double centering means that each distance is reduced by the corresponding row mean and column mean, and then the overall mean is added.

[0082] and Let the distance variance be defined as follows:

[0083]

[0084] Where N represents the number of samples, and They represent and The elements of the sample distance matrix after doubly centering.

[0085] The total information bottleneck loss for both modes is:

[0086]

[0087] in, This represents the total information bottleneck loss. and These represent the information bottleneck losses of the image modality and the genome modality, respectively. and These represent the intermediate layers of the neural network during training for the image modality and the genome modality, respectively.

[0088] Step S5: Dynamic weighted fusion of multimodal information. For example... Figure 2 As shown, unlike traditional simple splicing or averaging fusion modal feature schemes, this invention designs a dynamic weighted fusion module to adaptively adjust the weight of each modality based on the prediction uncertainty of each sample. Specifically, the pathological image and genomic features processed in steps S1 and S2 are first defined as follows: and The uncertainties of pathological and gene modalities are extracted separately and expressed as follows:

[0089]

[0090] in, and These represent the uncertainty parameters for pathological features and genetic features, respectively. It is an activation function. and These are uncertainty estimation networks for pathological and genetic modalities, respectively.

[0091] Then, the pathological and genomic features are input into the cross-modal joint expert, and the uncertainty of the cross-modal joint features is calculated, expressed as:

[0092]

[0093] in, Indicates cross-modal joint experts, This represents the combined characteristics of pathological and genomic modalities. This indicates the uncertainty of cross-modal joint features. This represents an uncertainty estimation network for cross-modal joint features. The uncertainty estimation network is a two-layer MLP network that includes Dropout.

[0094] The modal features and their uncertainties are then input into the hypernetwork to dynamically generate routing parameters for the current patient:

[0095]

[0096] in, This indicates a feature concatenation operation. Indicates a dynamic routing network. This indicates that the routing parameters are dynamically generated based on the current patient's modal characteristics and their uncertainties, and the dynamic routing network adopts a hypernetwork structure.

[0097] The routing scores for pathologists, genomics experts, and cross-modal joint experts are calculated based on dynamic routing parameters:

[0098]

[0099] in, , , Let represent the routing scores of pathology experts, genomics experts, and cross-modal joint experts, respectively. Considering the uncertainty associated with each expert, a penalty is applied to the routing scores, and a dynamic fusion weight is obtained using the Softmax function.

[0100]

[0101] in, , , These represent the dynamic fusion weights of pathologists, genomics experts, and cross-modal collaborative experts, respectively. This represents the uncertainty penalty coefficient. Finally, the three expert features are weighted and summed according to the dynamic fusion weights to obtain the final fused feature:

[0102]

[0103] in, As a pathology expert, As a genetics expert, As a cross-modal joint expert, pathology experts, gene experts and cross-modal joint experts together form a Mixture-of-Experts system. The network structure of pathology experts, gene experts and cross-modal joint experts are all two-layer MLP (Multi-Layer Perceptron) networks containing GELU activation function and batch normalization.

[0104] Step S6: Survival prediction loss. The obtained fused cross-modal representation... The input is fed into a fully connected layer with a Sigmoid activation function to predict the risk function. The negative log-likelihood loss is used as the supervision loss function, as shown below:

[0105]

[0106] in, and Let represent the risk function and the survival function, respectively. The total loss is obtained by adding the supervision loss to the total information bottleneck loss:

[0107]

[0108] The method of the present invention will be experimentally verified using specific data below:

[0109] To validate the effectiveness of the method, experiments were conducted on two publicly available cancer datasets from The Cancer Genome Atlas (TCGA): the Lung Adenocarcinoma (LUAD) dataset and the Bladder Urothelial Carcinoma (BLCA) dataset. These datasets contain paired whole-section pathological images and genomic data with labeled survival results. The LUAD dataset contains 453 cases, and the BLCA dataset contains 373 cases.

[0110] This embodiment of the experiment is implemented using the PyTorch deep learning framework with an NVIDIA A100 GPU, employing the Adam optimizer, and setting the initial learning rate to 2×10⁻. 4 The weight decay coefficient is 1×10⁻ 5 Considering the large size of a single pathological image, the batch size was set to 5 during training, and a total of 50 rounds of training iterations were completed. Experimental evaluation used 5-fold cross-validation, with the training and validation sets divided in a 4:1 ratio. The cross-validation consistency index (C-Index) and the area under the time-dependent curve (tAUC) were used as performance indicators, with higher values ​​for these two indicators indicating better model performance.

[0111] Table 1 shows the test results of the proposed method (IBDW) and other survival prediction algorithms on the LUAD and BLCA datasets. As can be seen from the table, the proposed method outperforms other methods in both C-index and tAUC metrics. The test results of other methods are not ideal, mainly due to insufficient multimodal reliability and rigid fusion issues in existing methods. In contrast, the proposed method achieves excellent results, with all metrics significantly superior to other methods. This reflects that the proposed method can effectively solve the problems of insufficient multimodal reliability and rigid fusion in existing methods.

[0112] Table 1. Comparison of evaluation metrics between the method of this invention and other methods on the LUAD and BLCA datasets.

[0113]

[0114] Note: Bold data indicates the optimal result.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal survival prediction method based on information bottleneck and dynamic weighted fusion, characterized in that, The method includes: Step S1, Pathological image modeling: Define the pathological image as a directed graph, and use a graph neural network to learn graph features to obtain pathological representations; Step S2, Genomic Data Modeling: Screen differentially expressed genes, analyze the pairwise correlation of the screened genes, binarize the adjacency matrix to generate the edge matrix, and finally input the genomic graph network into the graph attention network layer to construct a multi-layer graph representation; Step S3, Survival prediction task modeling: Format patient data and define survival prediction risk function and survival function; Step S4, Intramodal Information Compression Based on Information Bottleneck: An information bottleneck mechanism is introduced within the modality to refine the representation of each graph; Step S5, Dynamic weighted fusion of multimodal information: Adaptively adjust the weight of each modality based on the prediction uncertainty of each sample; Step S6, Survival prediction loss: Add the supervision loss to the total information bottleneck loss to obtain the total survival prediction loss.

2. The method according to claim 1, characterized in that, Step S1 further includes: The pathological images were cropped into a set of 256×256 image blocks. ,in, For image blocks, Given the total number of image patches, a 512-dimensional representation of each image patch is extracted using the pre-trained model PLIP, resulting in... ,in, For eigenvalues; Each image patch is then defined as a directed graph. , among which, nodes Corresponding image patch representation, edge The connectivity is dynamically updated during training. The graph features are learned using a sampling-aggregation approach with GraphSAGE graph neural networks. For the first ( Nodes in layer The representation, each layer first through Nearest neighbor node found direct neighborhood , For neighborhood set, Represent any neighboring node in the neighborhood set, and then use the aggregation function. Aggregate neighborhood representations into Then represent the current node. and The concatenated input includes a non-linear activation function. The fully connected layer is updated in the following way: in, For the first Layer learnable parameters For splicing operations; The design employs residual mapping and implements dense connections from the output of each layer to the last layer. Finally, it adaptively calculates the weighted sum of features of all nodes to obtain the pathological characterization.

3. The method according to claim 1, characterized in that, Step S2 further includes: Before screening from the patient cohort Identify differentially expressed genes associated with prognosis, and then use weighted gene co-expression network analysis to calculate the results after screening. The pairwise correlations of each gene are used to obtain the adjacency matrix. Then, by adjusting the adjacency matrix Binarization generates the edge matrix. Finally, the genome map network A multi-layer graph representation is constructed by inputting a 3-layer graph attention network (GAT layer). The node features of these multi-layer representations are then projected and concatenated to obtain a genome representation that simultaneously preserves individual gene expression information and inter-gene co-expression relationships. The genome graph network is used as an example. nodes For a single patient Gene expression values, with sides as .

4. The method according to claim 2 or 3, characterized in that, Step S3 further includes: Each patient's data is represented as a quadruple. ,in, and These represent the pathological images processed in step S1 and the genomic data processed in step S2, respectively. This is a right censoring state, indicating whether the resulting event was observed. Indicates observation, This indicates that it was not observed. This is the total survival time, divided into n intervals, each interval represented as... Survival prediction risk function Defined as an interval The probability of failure within, It is expressed as follows: in, This represents the random time at which a patient's survival outcome event occurs. The learnable parameters of the survival model are represented. Indicates pathology and genome Patient survival-related joint feature representations learned from data; Survival function The sequence risk used to estimate the outcome event is obtained by taking the negative of the cumulative risk function, and is expressed as: in, For survival function, Represents a variable.

5. The method according to claim 4, characterized in that, Step S4 further includes: The information bottleneck mechanism is represented as follows: in, The original input data, This represents the feature representation of the intermediate layer of a neural network. The label value is the patient's survival time. The balancing coefficient, dCor, represents the distance correlation coefficient, which is defined as: in, It is to prevent positive numbers with a denominator of zero. express and The distance covariance between them is defined as: Where N represents the number of samples, and These represent the input variables respectively. and intermediate features The elements of the sample distance matrix after doubly centering and They are defined as follows: The total information bottleneck loss for both modes is: in, This represents the total information bottleneck loss. and These represent the information bottleneck losses of the image modality and the genome modality, respectively. and These represent the intermediate layers of the neural network during training for the image modality and the genome modality, respectively.

6. The method according to claim 5, characterized in that, Step S5 further includes: The pathological images and genomic features processed in steps S1 and S2 are set as follows: and The uncertainty in extracting pathological images and gene modalities is represented as: in, and These represent the uncertainty parameters for pathological features and genetic features, respectively. For activation function, and These are uncertainty estimation networks for pathological and genetic modalities, respectively. Pathological and genomic features are input into a cross-modal joint expert, and the uncertainty of the cross-modal joint features is calculated, expressed as: in, Indicates cross-modal joint experts, This represents the combined characteristics of pathological and genomic modalities. This indicates the uncertainty of cross-modal joint features. An uncertainty estimation network representing cross-modal joint features; The modal features and their uncertainties are then input into the hypernetwork to dynamically generate routing parameters for the current patient: in, This indicates a feature concatenation operation. Indicates a dynamic routing network. This refers to routing parameters dynamically generated based on the current patient's modal characteristics and their uncertainties; Calculate routing scores for pathologists, genomics experts, and cross-modal joint experts based on dynamic routing parameters: in, , , Let represent the routing scores of pathology experts, genomics experts, and cross-modal joint experts, respectively. Considering the uncertainty associated with each expert, a penalty is applied to the routing scores, and a dynamic fusion weight is obtained using the Softmax function. in, , , These represent the dynamic fusion weights of pathologists, genomics experts, and cross-modal collaborative experts, respectively. This represents the uncertainty penalty coefficient. Finally, the three expert features are weighted and summed according to the dynamic fusion weights to obtain the final fused feature: in, As a pathology expert, As a genetics expert, For cross-modal joint experts.

7. The method according to claim 6, characterized in that, The supervision loss in step S6 is expressed as: in, and Let these represent the risk function and the survival function, respectively. The total loss is: .

Images