A Feature Reconstruction-Based Method for Predicting Clinical Multimodal Cancer Drug Response
By combining the TransP-Net model with the clinical-genomic feature completion module, the problems of biological differences and missing data in DLBCL drug response prediction are solved, enabling deep interaction of multimodal information and efficient prediction, thus improving prediction accuracy and applicability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- LIAONING NORMAL UNIVERSITY
- Filing Date
- 2025-11-17
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies for predicting drug response in DLBCL suffer from issues such as biological differences, insufficient utilization of single-modal features, and data gaps, resulting in insufficient reliability and applicability of prediction results in clinical translation.
Gene features were extracted using the TransP-Net model, and multimodal data were processed by a clinical information encoder. Pseudo-genomic features were generated through a clinical-genomic feature imputation module, and deep fusion was performed using a cross-attention mechanism to construct a hybrid contrastive binary loss function optimization model.
It improves prediction accuracy and robustness, effectively solves the problem of missing gene data, enhances feature representation capabilities, realizes deep interaction of multimodal information and clinical applicability, and is suitable for drug response prediction in real clinical environments.
Smart Images
Figure CN121528291B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of clinical medical technology, and in particular relates to a method for predicting clinical multimodal cancer drug response based on feature reconstruction. Background Technology
[0002] Diffuse large B-cell lymphoma (DLBCL) is the most common and heterogeneous subtype of non-Hodgkin lymphoma (NHL), accounting for 30%–40% of all new cases. DLBCL patients exhibit significant differences in clinical presentation and molecular pathological characteristics. Although the R-CHOP regimen (rituximab, cyclophosphamide, doxorubicin, vincristine, and prednisone) has significantly improved the overall survival of DLBCL patients, a considerable proportion still experience drug resistance, relapse, and even adverse reactions during treatment. Therefore, accurately predicting drug response in DLBCL has become an important research area in precision oncology. Currently, computational methods are increasingly advantageous in predicting drug response. Existing research can be divided into two categories: prediction methods based on public drug response databases and prediction methods based on clinical data.
[0003] Drug response prediction methods based on public databases have large-scale data and are easy to validate, but they have two limitations: First, there are significant biological differences between in vitro models such as cell lines and clinical patients, which limits the reliability of the prediction results in clinical translation. Second, most of these methods rely only on single-modal features such as genomics, while in clinical practice, patients' drug responses are often determined by multiple factors, including genes, pathways, pathological and immunological indicators, imaging results, and basic demographic information. Ignoring the comprehensive use of multimodal features not only makes it difficult to fully reflect the complexity of the disease, but also weakens the clinical applicability of the prediction model.
[0004] Drug response prediction based on clinical data also faces multiple challenges: First, the data sources are complex, with significant differences in characteristics across different types such as clinical, imaging, and pathological data. Direct fusion can easily introduce noise and weaken the predictive power of important modalities. Second, gene sequencing data is often missing in real-world clinical settings, with some patients lacking crucial genomic information, which limits the model's predictive capabilities. Therefore, how to fully utilize multimodal clinical information while effectively compensating for missing sequencing data and improving the stability and applicability of predictive models has become an urgent problem to be solved. Summary of the Invention
[0005] The purpose of this invention is to provide a clinical multimodal cancer drug response prediction method based on feature reconstruction, aiming to solve the problems mentioned in the background art.
[0006] The present invention is implemented as follows: a clinical multimodal cancer drug response prediction method based on feature reconstruction includes the following steps:
[0007] Step 1: Sequencing the input gene matrix and extracting gene features using the TransP-Net model;
[0008] Step 2: Process multimodal clinical data using a clinical information encoder and map it to a unified semantic space;
[0009] Step 3: Using the Clinical-Genomic Feature Imputation Module (CGIM), pseudo-genomic features are generated based on multimodal clinical features to address missing gene data;
[0010] Step 4: Using a conditional selection mechanism, select whether to use real or pseudo-gene features based on the existence of gene data. Then, splice and reconstruct the selected gene features with multimodal clinical features, and perform deep fusion through a cross-attention mechanism.
[0011] Step 5: Construct a hybrid contrastive binary loss function and optimize model training.
[0012] In a further technical solution, step 1 includes the following specific steps:
[0013] First, the input gene matrix is sequenced. Given a gene matrix... Each sample contains Individual gene characteristic values, Represents the set of real numbers. This represents the number of samples, which are then reconstructed into a feature sequence. :
[0014] ;
[0015] in, Indicates reconstruction;
[0016] This sequence serves as input to Trans-PNet, capturing potential interactions between genes through a self-attention mechanism; specifically, a query matrix is generated through a linear transformation. Key matrix Sum matrix :
[0017] ;
[0018] ;
[0019] ;
[0020] in, For learnable parameters, The dimension representing the feature; attention weights The calculation is as follows:
[0021] ;
[0022] in, This represents the attention mechanism. Represents the normalization function.
[0023] To enhance feature representation capabilities, residual connectivity and dynamic normalization (DyT) are introduced:
[0024] ;
[0025] ;
[0026] ;
[0027] in, This represents the features output through the attention mechanism and reconstructed through linear transformation. Represents a linear transformation. This indicates an enhanced feature that has undergone a dynamic normalization layer and retains the original information. Indicates a dynamic normalization layer. Indicates input, , and Indicates trainable parameters;
[0028] To further incorporate prior biological knowledge, a sparse deep neural network, P-Net, is used to explicitly model known biological pathway relationships; The layer output is:
[0029] ;
[0030] ;
[0031] in, This is a binary mask matrix that retains only the relationships between biological entities defined in the knowledge base. Indicates the first The weight matrix of the layer, Indicates the first Layer input, Indicates the first Layer bias vector, This represents the activation function. Indicates the first The output of the layer.
[0032] In a further technical solution, step 2 includes the following specific steps:
[0033] To integrate imaging reports, pathological indicators, and clinical information, a clinical information encoder was designed to map the features of each modality to a unified semantic space; the text of the imaging report is denoted as... Pathological and immunological indicators are Clinical information is recorded as ,in, Characteristic dimensions representing pathological and immune indicators, Characteristic dimensions representing clinical information;
[0034] First, for the radiological report text, a pre-trained BERT model is used to extract features:
[0035] ;
[0036] ;
[0037] in, The embedding representation sequence is at the word level. For sequence length, For compressed image text feature representation; for image text features Pathological and immune indicators and clinical information Each input is a separate DyT layer for normalization and alignment:
[0038] ;
[0039] ;
[0040] ;
[0041] in, Representing the textual features of compressed images Output features after dynamic normalization layer Indicators of pathology and immunity Output features after dynamic normalization layer Indicates basic patient information Output features after dynamic normalization layer This represents a dynamic normalization layer for image text features. This represents a dynamic normalization layer representing pathological and immune marker characteristics. This represents a dynamic normalization layer representing the patient's basic information characteristics.
[0042] In a further technical solution, in step 3, the clinical-genomic feature imputation module reconstructs missing genomic features from multimodal clinical features based on a multilayer perceptron:
[0043] ;
[0044] in, The generated pseudo-genomic features are consistent with the dimensions of real gene features; Indicates feature splicing, This represents the mapping function for the clinical-genomic feature completion module.
[0045] In a further technical solution, step 4 includes the following specific steps:
[0046] To dynamically handle the presence or absence of gene data, a conditional selection mechanism is introduced:
[0047] ;
[0048] in, This serves as an indicator of the existence of genomic data, indicating whether genomic data exists in the sample. Indicates the characteristic to which the value is assigned. Indicates genomic characteristics, Indicates pseudogenomic characteristics;
[0049] Next, genomic features and multimodal clinical features are spliced and reconstructed:
[0050] ;
[0051] ;
[0052] ;
[0053] in, express , and Clinical features after splicing express Reconstructed genomic features, express Reconstructed clinical characteristics Dimensions representing genomic features Dimensions representing clinical features;
[0054] To further achieve deep interaction, a cross-attention mechanism is adopted, using genomic features as queries and multimodal clinical features as keys and values:
[0055] ;
[0056] ;
[0057] ;
[0058] ;
[0059] in, This indicates the characteristics of interleaved attention. This indicates cross attention;
[0060] Further optimize the fused features using residual connections and a multilayer perceptron:
[0061] ;
[0062] ;
[0063] ;
[0064] in, Indicates the residual and reconstruction Features express Features processed by multilayer perceptron and dynamic normalization Indicates through residuals and the output features processed by the multilayer perceptron, Represents a multilayer perceptron;
[0065] The final output is the predicted probability of drug response.
[0066] A further technical solution is that, in step 5, the hybrid contrastive binary loss function includes binary cross-entropy loss and feature similarity loss:
[0067] ;
[0068] in, and Let these represent the true label and predicted probability of the i-th sample, respectively. This represents the binary cross-entropy loss;
[0069] ;
[0070] in, The number of samples with actual gene sequencing data. Represents the cosine similarity function. This represents the feature similarity loss;
[0071] The total loss is:
[0072] ;
[0073] in, These are the experimentally verified optimal weights; This represents the weighted total loss function value.
[0074] A further technical solution is provided in step 5. Set it to 0.6.
[0075] The present invention provides a clinical multimodal cancer drug response prediction method based on feature reconstruction, the beneficial effects of which are as follows:
[0076] (1) Improve prediction accuracy and robustness: By integrating multimodal clinical data and gene features and introducing a gene feature compensation mechanism, the limitations of a single data source are effectively overcome, and high prediction accuracy and stability can still be maintained in real clinical scenarios with missing and unbalanced data.
[0077] (2) Enhanced feature representation capability: Trans-PNet is used in combination with self-attention mechanism and biological pathway prior to achieve dual feature extraction at the gene and pathway levels, effectively capturing complex interactions between genes and multi-level biological semantic information, thereby improving the feature representation quality and interpretability.
[0078] (3) Effectively solve the problem of missing gene data: The CGIM module generates high-quality pseudo-gene features based on multimodal clinical features and combines feature comparison learning for constraints, so that the model can still maintain reliable prediction performance when gene sequencing data is lacking, which significantly expands the applicable population of the model.
[0079] (4) Optimize the effect of multimodal information fusion: adopt a cross-attention mechanism with genes as the query to ensure the dominant role of gene information in the multimodal fusion process, realize the deep and effective interaction between gene features and clinical information, and further improve the accuracy of prediction results.
[0080] (5) Improve clinical applicability and practicality: The overall model design fully considers the data characteristics and workflow in the real clinical environment, and can handle both complete gene data and missing gene data at the same time, which has good clinical translation prospects and application value. Attached Figure Description
[0081] Figure 1 This is a framework diagram of a clinical multimodal cancer drug response prediction method based on feature reconstruction provided in an embodiment of the present invention;
[0082] Figure 2 This is a framework diagram of Trans-PNet;
[0083] Figure 3 The results of the algorithm comparison experiments on the DLBCL multimodal dataset are presented. Detailed Implementation
[0084] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0085] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.
[0086] like Figure 1 and Figure 2 As shown in the illustration, this invention provides a feature-reconstruction-based clinical multimodal cancer drug response prediction method. This method constructs a clinical multimodal model (ClinMulti-DLBCL) for predicting drug response in diffuse large B-cell lymphoma, aiming to achieve accurate drug response prediction by integrating gene sequencing and clinical multimodal data. The model employs an end-to-end multi-stage processing flow: first, gene features are extracted using TransP-Net, and multimodal clinical data is processed using a clinical information encoder; then, pseudo-gene features are generated using a clinical-genome imputation module to address data gaps; finally, multimodal deep fusion is achieved using a clinical information decoder to output the prediction results. Specifically, the method includes the following steps:
[0087] Step 1: Gene sequencing and Trans-PNet modeling;
[0088] Genetic data is of great value in drug response prediction, but traditional methods struggle to effectively capture its high-dimensional features and the complex relationships between biological pathways. Therefore, this method first performs sequence processing on the input gene matrix, given the gene matrix... Each sample contains Individual gene characteristic values, Represents the set of real numbers. This represents the number of samples, which are then reconstructed into a feature sequence. :
[0089] ;
[0090] in, Indicates reconstruction;
[0091] This sequence serves as input to Trans-PNet, capturing potential interactions between genes through a self-attention mechanism; specifically, a query matrix is generated through a linear transformation. Key matrix Sum matrix :
[0092] ;
[0093] ;
[0094] ;
[0095] in, For learnable parameters, The dimension representing the feature; attention weights The calculation is as follows:
[0096] ;
[0097] in, This represents the attention mechanism. Represents the normalization function.
[0098] To enhance feature representation capabilities, residual connections and dynamic normalization are introduced:
[0099] ;
[0100] ;
[0101] ;
[0102] in, This represents the features output through the attention mechanism and reconstructed through linear transformation. Represents a linear transformation. This indicates an enhanced feature that has undergone a dynamic normalization layer and retains the original information. Indicates a dynamic normalization layer. Indicates input, , and This represents the trainable parameters.
[0103] To further incorporate prior biological knowledge, this method combines a sparse deep neural network, P-Net, to explicitly model known biological pathway relationships. The layer output is:
[0104] ;
[0105] ;
[0106] in, This is a binary mask matrix that retains only the relationships between biological entities defined in knowledge bases such as Reactome. Indicates the first The weight matrix of the layer, Indicates the first Layer input, Indicates the first Layer bias vector, This represents the activation function. Indicates the first The output of the layer.
[0107] Step 2: Multimodal clinical information encoding and alignment;
[0108] To integrate multi-source heterogeneous data such as imaging reports, pathological indicators, and clinical information, a clinical information encoder was designed to map features from each modality to a unified semantic space. The text of the imaging report is denoted as... Pathological and immunological indicators are Clinical information is recorded as ,in, Characteristic dimensions representing pathological and immune indicators, Characteristic dimensions representing clinical information.
[0109] First, for the radiological report text, a pre-trained BERT model is used to extract features:
[0110] ;
[0111] ;
[0112] in, The embedding representation sequence is at the word level. For sequence length, This is a compressed representation of image text features. (The text then discusses image text features.) Pathological and immune indicators and clinical information Each input is a separate DyT layer for normalization and alignment:
[0113] ;
[0114] ;
[0115] ;
[0116] in, Representing the textual features of compressed images Output features after dynamic normalization layer Indicators of pathology and immunity Output features after dynamic normalization layer Indicates basic patient information Output features after dynamic normalization layer This represents a dynamic normalization layer for image text features. This represents a dynamic normalization layer representing pathological and immune marker characteristics. This represents a dynamic normalization layer representing the patient's basic information characteristics.
[0117] Step 3: Construct a clinical-genomic feature completion module;
[0118] To address the issue of missing genomic data, a clinical-genomic feature imputation module was constructed, which reconstructs missing genomic features from multimodal clinical features based on a multilayer perceptron.
[0119] ;
[0120] in, The generated pseudo-genomic features are consistent with the dimensions of real gene features; Indicates feature splicing, This represents the mapping function for the clinical-genomic feature completion module.
[0121] Step 4: Condition selection and cross-modal fusion;
[0122] To dynamically handle the presence or absence of gene data, a conditional selection mechanism is introduced:
[0123] ;
[0124] in, This serves as an indicator of the existence of genomic data, indicating whether genomic data exists in the sample. Indicates the characteristic to which the value is assigned. Indicates genomic characteristics, This indicates pseudogenomic characteristics.
[0125] Next, genomic features and multimodal clinical features are spliced and reconstructed:
[0126] ;
[0127] ;
[0128] ;
[0129] in, express , and Clinical features after splicing express Reconstructed genomic features, express Reconstructed clinical characteristics Dimensions representing genomic features Dimensions representing clinical features.
[0130] To further achieve deep interaction, a cross-attention mechanism is adopted, using genomic features as queries and multimodal clinical features as keys and values:
[0131] ;
[0132] ;
[0133] ;
[0134] ;
[0135] in, This indicates the characteristics of interleaved attention. This indicates cross attention;
[0136] Further optimize the fused features using residual connections and a multilayer perceptron:
[0137] ;
[0138] ;
[0139] ;
[0140] in, Indicates the residual and reconstruction Features express Features processed by multilayer perceptron and dynamic normalization Indicates through residuals and the output features processed by the multilayer perceptron, This represents a multilayer perceptron, whose final output is the predicted probability of drug response.
[0141] Step 5: Hybrid contrastive binary loss function;
[0142] To optimize model training, a hybrid contrastive binary loss function is designed, including binary cross-entropy loss and feature similarity loss:
[0143] ;
[0144] in, and Let these represent the true label and predicted probability of the i-th sample, respectively. This represents the binary cross-entropy loss.
[0145] ;
[0146] in, The number of samples with actual gene sequencing data. Represents the cosine similarity function. This represents the feature similarity loss.
[0147] The total loss is:
[0148] ;
[0149] in, The weight is set to 0.6, which is the optimal weight verified by experiments. This represents the weighted total loss function value.
[0150] Based on the above methods, data from patients with pathologically confirmed diffuse large B-cell lymphoma from the hematology department of a large tertiary hospital in China were validated. All patients had received at least one cycle of R-CHOP or a modified regimen of chemotherapy and underwent PET-CT examinations before and after treatment. Treatment response was assessed using the Deauville five-point scale: 1-3 points for complete remission and 4-5 points for incomplete remission.
[0151] Gene sequencing data covers DLBCL hotspot genes such as CD79B, MYD88, and TP53. The gene mutation load is quantified by calculating the cumulative mutation intensity (CMI), which combines the mutation ratio and the probability of disease, and can more accurately reflect the harmful effects of gene mutations.
[0152] Multimodal clinical data included: (1) PET-CT reports; (2) pathological and immunohistochemical indicators; (3) lymphocyte subsets and cytokine levels detected by flow cytometry; and (4) basic clinical information such as age and gender. All pathological and flow cytometry data indicators were standardized into three levels: high, normal, or low to ensure data consistency and processability.
[0153] Finally, the ClinMulti-DLBCL model was subjected to 5-fold cross-validation experiments on the DLBCL clinical multimodal dataset and compared with four algorithms (SVM, MLP, DeepTTC, and BANDRP). Figure 3 The performance results of CGB-DLBCR and the comparative models are presented. Experimental results show that CGB-DLBCR significantly outperforms all comparative algorithms in key metrics such as accuracy, precision, AUC, F1 score, and Aupr. This demonstrates that CGB-DLBCR can maintain high recall while achieving a balance between precision and F1 score in real-world clinical scenarios with missing gene data and class imbalance, exhibiting excellent robustness and generalization ability.
[0154] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A clinical multi-modality cancer drug response prediction method based on feature reconstruction, characterized in that, Includes the following steps: Step 1: Sequencing the input gene matrix and extracting gene features using the TransP-Net model; Step 2: Process multimodal clinical data using a clinical information encoder and map it to a unified semantic space; Step 3: Using the clinical-genomic feature imputation module, pseudo-genomic features are generated based on multimodal clinical features to address missing gene data; Step 4: Using a conditional selection mechanism, select whether to use real or pseudo-gene features based on the existence of gene data. Then, splice and reconstruct the selected gene features with multimodal clinical features, and perform deep fusion through a cross-attention mechanism. Step 5: Construct a hybrid contrastive binary loss function and optimize model training; Step 1 includes the following specific steps: First, the input gene matrix is serialized, and the gene matrix is given where each sample contains gene feature values, denotes the real set, denotes the number of samples, and it is reconstructed into a feature sequence : ； wherein represents reconstitution; The sequence is input to the Trans-PNet, which captures potential interactions between genes through a self-attention mechanism; specifically, a query matrix , key matrix , and value matrix are generated through linear transformations: ； ； ； wherein, are learnable parameters, denotes the representation dimension of the feature; attention weight is calculated as follows: ； wherein, denotes an attention mechanism, denotes a normalization function, To enhance feature representation capabilities, residual connections and dynamic normalization are introduced: ； ； ； in, This represents the features output through the attention mechanism and reconstructed through linear transformation. Represents a linear transformation. This indicates an enhanced feature that has undergone a dynamic normalization layer and retains the original information. Indicates a dynamic normalization layer. Indicates input, , and Indicates trainable parameters; To further incorporate prior biological knowledge, a sparse deep neural network, P-Net, is used to explicitly model known biological pathway relationships; The layer output is: ； ； in, This is a binary mask matrix that retains only the relationships between biological entities defined in the knowledge base. Indicates the first The weight matrix of the layer, Indicates the first Layer input, Indicates the first The layer's bias vector, This represents the activation function. Indicates the first The output of the layer, Indicates genomic characteristics; In step 3, the clinical-genomic feature imputation module reconstructs missing genomic features from multimodal clinical features based on a multilayer perceptron: ； in, The generated pseudo-genomic features are consistent with the dimensions of real gene features; Indicates feature splicing, This represents the mapping function for the clinical-genomic feature completion module. Representing the textual features of compressed images Output features after dynamic normalization layer Indicators of pathology and immunity Output features after dynamic normalization layer Indicates basic patient information Output features after dynamic normalization layer.
2. The clinical multimodal cancer drug response prediction method based on feature reconstruction according to claim 1, characterized in that, Step 2 includes the following specific steps: To integrate imaging reports, pathological indicators, and clinical information, a clinical information encoder was designed to map the features of each modality to a unified semantic space; the text of the imaging report is denoted as... Pathological and immunological indicators are Clinical information is recorded as ,in, Characteristic dimensions representing pathological and immune indicators, Characteristic dimensions representing clinical information; First, for the radiological report text, a pre-trained BERT model is used to extract features: ； ； in, The embedding representation sequence is at the word level. For sequence length, For compressed image text feature representation; for image text features Pathological and immune indicators and clinical information Each input is a separate DyT layer for normalization and alignment: ； ； ； in, This represents a dynamic normalization layer for image text features. This represents a dynamic normalization layer representing pathological and immune marker characteristics. This represents a dynamic normalization layer representing the patient's basic information characteristics.
3. The clinical multimodal cancer drug response prediction method based on feature reconstruction according to claim 2, characterized in that, Step 4 includes the following specific steps: To dynamically handle situations where gene data is present, a conditional selection mechanism is introduced: ； in, This serves as an indicator of the existence of genomic data, indicating whether genomic data exists in the sample. Indicates the characteristic to which the value is assigned. Indicates pseudogenomic characteristics; Next, genomic features and multimodal clinical features are spliced and reconstructed: ； ； ； in, express , and Clinical features after splicing express Reconstructed genomic features, express Reconstructed clinical characteristics Dimensions representing genomic features Dimensions representing clinical features; To further achieve deep interaction, a cross-attention mechanism is adopted, using genomic features as queries and multimodal clinical features as keys and values: ； ； ； in, This indicates the characteristics of interleaved attention. This indicates cross attention; Further optimize the fused features using residual connections and a multilayer perceptron: ； ； ； in, Indicates the residual and reconstruction Features express Features processed by multilayer perceptron and dynamic normalization Indicates through residuals and the output features processed by the multilayer perceptron, Represents a multilayer perceptron; The final output is the predicted probability of drug response.
4. The clinical multimodal cancer drug response prediction method based on feature reconstruction according to claim 3, characterized in that, In step 5, the hybrid contrastive binary loss function includes binary cross-entropy loss and feature similarity loss: ； in, and Let these represent the true label and predicted probability of the i-th sample, respectively. This represents the binary cross-entropy loss; ； in, The number of samples with actual gene sequencing data. Represents the cosine similarity function. This represents the feature similarity loss; The total loss is: ； in, These are the experimentally verified optimal weights; This represents the weighted total loss function value.
5. The clinical multimodal cancer drug response prediction method based on feature reconstruction according to claim 4, characterized in that, In step 5, Set it to 0.6.