A method for predicting drug response using high-dimensional small sample data

Abstract

This invention relates to the field of bioinformatics, and in particular to a method for predicting drug response using high-dimensional, small-sample data. The method includes: constructing a knowledge injection model, which utilizes feature embeddings extracted from a biomedical knowledge graph as additional input, and introduces a diversified sparse feature extraction module and a local alignment context learning submodule. The diversified sparse feature extraction module extracts diverse high-level features from the high-dimensional data through sparse feature selection and selective feature extraction of gene features and their embeddings. The local alignment context learning submodule extracts potential information interactions between high-level feature embeddings through a neighborhood attention mechanism and maintains the original high-level feature information through a self-alignment strategy. This invention avoids overfitting while more effectively capturing important information in the data, significantly improving the accuracy of drug response prediction.

Description

Technical Field

[0001] This invention relates to the field of bioinformatics, and in particular to a method for predicting drug response using high-dimensional, small-sample data. Background Technology

[0002] Cancer is a complex genetic disease. Due to the heterogeneity of tumors, patients with the same type of cancer may experience varying outcomes when treated with the same drugs. Cancer drug response prediction can help patients develop personalized treatment plans, playing a crucial role in precision medicine.

[0003] In recent years, with the development of high-throughput sequencing technology, the amount of data in the biomedical field has grown rapidly, especially in genomics research, which typically generates high-dimensional gene expression data. This data contains tens of thousands of gene features, but the sample size is relatively small, posing a significant challenge to machine learning models. Traditional statistical methods, such as Least Absolute Shrinkage and Selection Operator (LASSO) and Principal Component Analysis (PCA), while performing well in dimensionality reduction and feature selection, are ineffective when dealing with complex nonlinear patterns. On the other hand, decision trees and linear models such as XGBoost are relatively stable on small sample datasets, but when dealing with high-dimensional data, these models are prone to overfitting and struggle to accurately predict drug responses.

[0004] Neural networks perform well with abundant data, but their application in high-dimensional, small-sample scenarios is often limited by model complexity and insufficient data volume, making them prone to overfitting. Furthermore, despite the wealth of domain-specific knowledge in the biomedical field, this information is not fully utilized in neural network models, resulting in a less profound and comprehensive understanding of the data. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem of insufficient accuracy in predicting drug response to high-dimensional small sample data in the prior art.

[0006] To address the aforementioned technical problems, this invention provides a method for predicting drug responses using high-dimensional, small-sample data, comprising:

[0007] A knowledge injection model is constructed, including a data preprocessing module, a diversified sparse feature extraction module, a multi-branch context learning module, and a multi-branch feature fusion and prediction module;

[0008] The data preprocessing module is used to extract gene features and their embeddings from the genomic dataset;

[0009] Each branch of the diversified sparse feature extraction module is used to perform sparse feature selection and selective feature extraction on gene features and their embeddings, resulting in high-level features and their embeddings for each branch.

[0010] The multi-branch context learning module includes a local alignment context learning submodule and inter-branch attention. The high-level feature embeddings of each branch are input into the local alignment context learning submodule. The neighborhood attention mechanism of the local alignment context learning submodule is used to extract the potential information interaction between the high-level feature embeddings, resulting in the feature embedding output by the neighborhood attention mechanism of each branch. Then, the cross-attention mechanism of the local alignment context learning submodule is used to align the high-level feature embeddings of each branch with the feature embeddings output by the neighborhood attention mechanism, resulting in the target feature embedding of each branch. Finally, the high-level features of each branch are input into the inter-branch attention mechanism to calculate the weight of each branch.

[0011] The high-level features and target features of each branch are embedded and fused using the multi-branch feature fusion and prediction module. Then, the weights of each branch are used to sum the results to obtain the fused features. The fused features are then input into the multilayer perceptron to obtain the drug response prediction results.

[0012] Preferably, the data preprocessing module is used to extract gene features x from publicly available genomic datasets. o And utilize biomedical knowledge graphs to obtain gene characteristics x o Embedded E o .

[0013] Preferably, each branch of the diversified sparse feature extraction module is used to perform sparse feature selection on gene features and their embeddings, as shown in the following formula:

[0014]

[0015] Among them, SM (i) (·) denotes the sparse feature selection of the i-th branch, i = 1, 2, ..., k, where k represents the total number of branches; E represents the sparse feature weight matrix of the i-th branch. o This represents the gene feature embedding, where x1 is the input for sparse feature selection, including gene features and gene feature embeddings;

[0016] All branches are related to gene trait x o After performing sparse feature selection, output the selected features X. s All branches embedding of gene features E o After performing sparse feature selection, the output is the feature embedding E after sparse feature selection. s .

[0017] Preferably, selective feature extraction is performed on the features and their embeddings after sparse feature selection to obtain the high-level features and their embeddings of each branch, as shown in the formula:

[0018]

[0019] Among them, FE (i) (·) represents selective feature extraction for the i-th branch. This represents the attention weighting matrix for the i-th branch. x1 represents the selective feature weight matrix of the i-th branch, and x2 is the input for selective feature extraction, including the features after sparse feature selection and their embeddings.

[0020] All branches select the sparse feature X. s After selective feature extraction, the high-level feature X is output. e ; Feature embedding E after sparse feature selection for all branches s After selective feature extraction, the high-level feature embedding E is output. e .

[0021] Preferably, the neighborhood attention mechanism of the local alignment context learning submodule is used to extract the potential information interaction between high-level feature embeddings, resulting in the feature embedding output by the neighborhood attention mechanism of each branch. The formula includes:

[0022]

[0023] in, For the high-level feature embedding of the i-th branch, For the QK operation of neighborhood attention on the i-th branch, Let V be the neighborhood attention operation on the i-th branch, where ρ(m) represents the index of the neighboring feature embeddings of the m-th high-level feature embedding, and m represents the number of features after selective feature extraction. This represents the m-th feature embedded in the high-level feature of the i-th branch. express neighborhood feature embedding, and All are linear transformation weights for the i-th branch. Let represent the bias of the m-th feature embedded in the high-level feature of the i-th branch; σ represents the softmax activation function. This represents the feature embedding output by the neighborhood attention mechanism of the i-th branch.

[0024] Preferably, the cross-attention mechanism of the local alignment context learning submodule is used to align the high-level feature embeddings of each branch with the feature embeddings output by the neighborhood attention mechanism, resulting in the target feature embedding for each branch, as shown in the formula:

[0025]

[0026] in, This represents the target feature embedding of the i-th branch. This represents the feature embedding output by the neighborhood attention mechanism of the i-th branch. For the high-level feature embedding of the i-th branch, W q1 、W k1 and W v1 Both represent the weights of cross-attention.

[0027] Preferably, the high-level features of each branch are input into the inter-branch attention, and the weight of each branch is calculated using the following formula:

[0028] w t =Softmax(init(X) e )w a )

[0029] Among them, w t Indicates the branch weight, init(X) e ) represents high-level feature X e The initialized fixed input weight information, w a The weight matrix represents the attention between branches.

[0030] Preferably, the high-level features and target features of each branch are embedded and fused, and then the fused features are obtained by weighted summation using the weights of each branch, as shown in the formula:

[0031]

[0032] Among them, X h For fusion features, k represents the total number of branches. Let w1 and w2 be the branch weights of the i-th branch, and w1 and w2 be the initial weights of the fused features. Embed the target features for the i-th branch. This represents the high-level feature of the i-th branch.

[0033] Preferably, the total loss for training the knowledge injection model includes mean squared error loss and total contrast loss.

[0034] Preferably, the formula for calculating the total contrast loss is:

[0035]

[0036] Among them, L c The total contrast loss is N, where N is the number of pairwise combinations between branches, and S represents the set of pairwise combinations of high-level features from different branches. and These represent the high-level features of the i-th branch and the j-th branch, respectively. This represents the high-level features of the z-th sample in the i-th branch. This represents the high-level features of the z-th sample in the j-th branch; Let represent the contrast loss between the i-th branch and the j-th branch with respect to the z-th sample;

[0037] The formula for calculating the contrast loss between two branches is:

[0038]

[0039] Where s represents the s-th sample in the training batch, and τ represents the temperature parameter. express and The similarity score between them.

[0040] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0041] This invention discloses a drug response prediction method for high-dimensional, small-sample data. By constructing a knowledge-injection model, it utilizes feature embeddings extracted from a biomedical knowledge graph as additional input and introduces a diversified sparse feature extraction module and a local alignment context learning submodule. The diversified sparse feature extraction module performs sparse feature selection and selective feature extraction on gene features and their embeddings, combined with contrastive loss constraints, to extract diverse high-level features from high-dimensional data, effectively reducing data dimensionality and extracting feature diversity and uniqueness. The local alignment context learning submodule extracts potential information interactions between high-level feature embeddings through a neighborhood attention mechanism and maintains the original high-level feature information through a self-alignment strategy. This invention avoids overfitting while more effectively capturing important information in high-dimensional, small-sample data, significantly improving the accuracy of drug response prediction. Attached Figure Description

[0042] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0043] Figure 1 This is a flowchart of a drug response prediction method for high-dimensional small sample data according to the present invention, wherein... Figure 1 In this context, 'a' represents the data preprocessing module. Figure 1 In this context, 'b' represents the diverse sparse feature extraction module. Figure 1 In this context, 'c' represents the multi-branch context learning module. Figure 1 In this context, d represents the multi-branch feature fusion and prediction module;

[0044] Figure 2 This is the performance comparison result of the present invention and existing methods on multiple datasets in Example 2. Detailed Implementation

[0045] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0046] Example 1

[0047] Reference Figure 1 As shown, this invention provides a method for predicting drug response using high-dimensional small sample data, comprising:

[0048] A knowledge injection model is constructed, including a data preprocessing module, a diversified sparse feature extraction module, a multi-branch context learning module, and a multi-branch feature fusion and prediction module; the multi-branch context learning module includes a local alignment context learning sub-module and inter-branch attention.

[0049] The data preprocessing module is used to extract gene features and their embeddings from the genomic dataset;

[0050] Each branch of the diversified sparse feature extraction module is used to perform sparse feature selection and selective feature extraction on gene features and their embeddings, resulting in high-level features and their embeddings for each branch.

[0051] The high-level feature embeddings of each branch are input into the local alignment context learning submodule. The neighborhood attention mechanism of the local alignment context learning submodule is used to extract the potential information interaction between the high-level feature embeddings, and the feature embedding output by the neighborhood attention mechanism of each branch is obtained. Then, the cross attention mechanism of the local alignment context learning submodule is used to align the high-level feature embeddings of each branch with the feature embeddings output by the neighborhood attention mechanism, and the target feature embedding of each branch is obtained. The high-level features of each branch are input into the inter-branch attention, and the weight of each branch is calculated.

[0052] The high-level features and target features of each branch are embedded and fused using the multi-branch feature fusion and prediction module. Then, the weights of each branch are used to sum the results to obtain the fused features. The fused features are then input into the multilayer perceptron to obtain the drug response prediction results.

[0053] The following section provides a detailed explanation of the processing procedures for each module in the knowledge injection model.

[0054] The data preprocessing module extracts gene features x from publicly available genomic datasets. o ∈R n×d Where n represents the number of samples and d represents the number of features; and gene features x are obtained using a biomedical knowledge graph (KG). o Embedded E o ∈R d×c , where c represents the dimension of the feature embedding.

[0055] This invention utilizes feature embeddings extracted from biomedical knowledge graphs (such as the association information between genes, diseases, and drugs) as additional input during the model's feature extraction and context learning processes. This knowledge injection mechanism effectively enhances the model's understanding of gene features and improves the accuracy of drug response prediction.

[0056] Gene features and their embeddings are input into a diverse sparse feature extraction module.

[0057] The diversified sparse feature extraction module selects important features from high-dimensional gene expression data by introducing sparsity mechanisms and contrastive constraints. Specifically, this module employs a multi-branch strategy, with each branch responsible for abstracting features from different perspectives, and a contrastive loss function ensures that the feature extraction from each branch has sufficient diversity. This diversified sparse feature extraction mechanism effectively reduces the dimensionality of the model input while retaining the most important features.

[0058] To efficiently extract effective feature information from high-dimensional gene expression data, the diversified sparse feature extraction module first utilizes a multi-branch strategy to perform sparse feature selection on gene features and their embeddings. The formula for sparse feature selection is as follows:

[0059]

[0060] Among them, SM (i) (·) denotes the sparse feature selection of the i-th branch, i = 1, 2, ..., k, where k represents the total number of branches; E represents the sparse feature weight matrix of the i-th branch. o This represents the gene feature embedding, where x1 is the input for sparse feature selection, including gene features and gene feature embeddings.

[0061] Sparse feature weight matrix It consists of multiple linear networks, with each layer sequentially undergoing LeakReLU activation, batch normalization, and dropout operations. The LeakReLU activation function is used to maintain network sparsity, and the dropout operation is used to randomly discard neurons.

[0062] All branches are related to gene trait xo After performing sparse feature selection, output the selected features X. s ∈R k ×d×n All branches embedding of gene features E o After performing sparse feature selection, the output is the feature embedding E after sparse feature selection. s ∈R k×d×c , represented as:

[0063] [X s E s ] = stack[SM (1) (T o ),SM (2) (T o ),...,SM (k) (T o )]

[0064] Among them, T o ∈R d×(n+c) This represents the combination of gene feature x0 and its embedding E0.

[0065] Then select the feature X after sparse features s and its embedded E s Selective feature extraction is performed. To effectively extract high-level features, a learnable weight matrix w is used. e1 ∈R k×m×d This process captures the collaborative interactions between selected features and performs dimensionality reduction, where m represents the number of extracted features. Simultaneously, an attention mechanism is applied to weight the selectively extracted features and their embeddings, thereby focusing on significantly effective high-level features. The attention weighting matrix for this mechanism is w. e0 ∈R k×m×d .

[0066] The formula for selective feature extraction is:

[0067]

[0068] Among them, FE (i) (·) represents selective feature extraction for the i-th branch. This represents the attention weighting matrix for the i-th branch. x1 represents the selective feature weight matrix of the i-th branch, and x2 is the input for selective feature extraction, including the features after sparse feature selection and their embeddings.

[0069] All branches select the sparse feature X. s After selective feature extraction, the high-level feature X is output. e ∈R k×m×n ; Feature embedding E after sparse feature selection for all branchess After selective feature extraction, the high-level feature embedding E is output. e ∈R k×m×c , represented as:

[0070]

[0071] Among them, T s ∈R k×d×(n+c) The feature X represents the sparse feature selection. s and its embedded E s The combination of .

[0072] To ensure the diversity of high-level features extracted from different branches, a contrastive loss constraint is used among the high-level features output by multiple branches of the diverse sparse feature extraction module during the training of the knowledge injection model. The specific steps are as follows:

[0073] Calculate the similarity scores between the high-level features output by different branches:

[0074]

[0075] in, This represents the high-level features of the z-th sample in the i-th branch. Let represent the high-level features of the z-th sample in the j-th branch, and sim denotes the similarity between features of different branches calculated through vector multiplication. express and The similarity score between them.

[0076] Calculate the contrast loss between each pair of branches, treating the corresponding sample similarity scores between branches as positive sample pairs, and the remaining samples as negative sample pairs.

[0077] The formula for calculating the contrast loss between two branches is:

[0078]

[0079] in, Let represent the contrast loss between the i-th branch and the j-th branch with respect to the z-th sample, b represent the total number of samples in the training batch, s represent the s-th sample in the training batch, and τ represent the temperature parameter.

[0080] Combine the contrast losses of all branch combinations to calculate the total contrast loss, using the following formula:

[0081]

[0082] Among them, L cThe total contrastive loss is N, where N is the number of pairwise combinations between branches, and S represents the set of pairwise combinations of high-level features from different branches. and These represent the high-level features of the i-th branch and the j-th branch, respectively.

[0083] During training, the weight matrix of each branch is optimized using the total contrastive loss, so that the high-level features extracted by each branch after training have diversity.

[0084] The diversified sparse feature extraction module employs a multi-branch strategy, combining diversified sparse feature extraction, selective feature extraction, and a contrastive loss constraint mechanism to extract high-level features from high-dimensional gene expression data. Diversified sparse feature extraction uses frozen feature embeddings, weighted networks, and a sigmoid activation function to assign weight scores to gene features and their embeddings in each branch, thus achieving feature selection. Selective feature extraction then uses two linear networks to extract features and their embeddings selected from the sparse feature selection. The output of one network uses a sigmoid activation function to apply attention weights to the features extracted by the other network, achieving selective feature extraction and obtaining high-level features and their embeddings from multiple branches. The contrastive loss constraint first calculates the similarity of high-level features between different branches, then uses a total contrastive loss to differentiate the similarity between the high-level features extracted from each branch, thus achieving the goal of diversifying high-level features and their embeddings.

[0085] High-level features and their embeddings are input into a multi-branch context learning module, where high-level feature embeddings are input into a local alignment context learning sub-module, and high-level features are input into inter-branch attention.

[0086] To further enhance the model's understanding of relationships between features, this invention proposes a local alignment context learning submodule. This module employs a neighborhood attention mechanism to capture the contextual relationships between high-dimensional features and uses a self-alignment mechanism to ensure the preservation of original feature information during the learning process. The local alignment context learning submodule improves the accuracy of feature representation by effectively fusing local information with contextual relationships.

[0087] The local alignment context learning submodule processes high-level feature embeddings as follows:

[0088] Due to the limited sample size, excessive context learning may introduce bias and lead to overfitting. Therefore, this invention uses a neighborhood attention (NA) mechanism to compute the comprehensive neighborhood information of the high-level feature embeddings of each branch in order to extract the potential interactions between the high-level feature embeddings.

[0089] The high-level feature embeddings of each branch are input into the neighborhood attention mechanism. The neighborhood attention mechanism is then used to extract the latent information interactions between the high-level feature embeddings, resulting in the feature embeddings output by the neighborhood attention mechanism for each branch. The formula includes:

[0090]

[0091] in, For the high-level feature embedding of the i-th branch, For the QK operation of neighborhood attention on the i-th branch, For the V operation of neighborhood attention on the i-th branch, ρ(m)∈R l This represents the index of the l neighboring feature embeddings of the m-th high-level feature embedding, where m represents the number of features after selective feature extraction. This represents the m-th feature embedded in the high-level feature of the i-th branch. express The embedding of l neighborhood features, and All are linear transformation weights for the i-th branch. Let represent the bias of the m-th feature embedded in the high-level feature of the i-th branch; σ represents the softmax activation function. Let c represent the feature embedding output by the neighborhood attention mechanism of the i-th branch, where c represents the dimension of the feature embedding.

[0092] Furthermore, to prevent misalignment issues between feature inputs and feature embeddings that may arise during subsequent multi-branch feature fusion, this invention also utilizes a self-alignment mechanism in the local alignment context learning submodule. The self-alignment mechanism compares the potential interactions between extracted high-level feature embeddings to the original high-level feature embeddings via a cross-attention mechanism, thereby preserving as much original information as possible and ensuring information consistency between high-level features and their embeddings.

[0093] The input and output of the neighborhood attention mechanism for each branch, namely the high-level feature embedding and the feature embedding output by the neighborhood attention mechanism, are aligned using a self-alignment mechanism to obtain the target feature embedding for each branch. The formula is as follows:

[0094]

[0095] in, W represents the target feature embedding of the i-th branch. q1 、W k1 and W v1 Both represent the weights of cross-attention.

[0096] The local alignment context learning submodule is able to capture detailed contextual information while preserving the original feature embeddings.

[0097] The process of inter-branch attention for processing high-level features is as follows:

[0098] Since the local alignment context learning submodule utilizes multiple branches to extract high-level features and learns the context information embedded in multiple branch features, this invention focuses on the importance of each branch, puts high-level features into a multi-layer linear network, and normalizes the output of the multi-layer linear network to between 0 and 1 through the softmax activation function. At the same time, the normalized results are summed to 1, thereby achieving the purpose of learning the importance of each branch.

[0099] The formula for calculating the weight of each branch is as follows:

[0100] w t =Softmax(init(X) e )w a )

[0101] Among them, w t ∈R k Indicates the branch weight, init(X) e )∈R k×m Representing high-level feature X e The initialized fixed input weight information; w a ∈R m×1 The weight matrix represents the attention between branches. It consists of multiple linear networks, with each layer of the linear network sequentially performing LeakReLU activation, batch normalization, and dropout operations.

[0102] In the final stage of the knowledge injection model, the features from different branches are weighted and fused. The fused features are then processed through a multilayer perceptron network to ultimately output a prediction of the drug response.

[0103] In the multi-branch feature fusion and prediction module, the high-level features and target features of each branch are embedded and fused element-wise by multiplication. Then, the fused features are obtained by weighted summation using branch weights. The formula is as follows:

[0104]

[0105] Among them, X h ∈R n×h For fusion features, k represents the total number of branches. Let w1, w2 ∈ R be the branch weight of the i-th branch. c×h The initial weights represent the fused features. Embed the target features for the i-th branch. This represents the high-level feature of the i-th branch.

[0106] The fused features are input into a multilayer perceptron to obtain the drug response prediction y∈R. n .

[0107] During training, the knowledge injection model uses mean squared error (MSE) loss to minimize the error between the predicted and true values, as shown in the formula:

[0108]

[0109] Among them, L r Let y be the mean squared error loss, b represent the total number of samples in the training batch, and y be the mean squared error loss. z and Let represent the predicted drug response value and the actual drug response value for the z-th sample, respectively.

[0110] The total loss of the knowledge injection model also uses contrastive loss to minimize the similarity between branches, as shown in the formula:

[0111] L = L r +αL c

[0112] Where L is the total loss, L r For mean square error loss, L c Let α be the total contrastive loss and α be the contrastive loss weight. The value of α ranges from [0,1]. Since the differences between branches become more significant during training, more attention is paid to the difference between the predicted value and the true value, i.e., the mean squared error loss. Therefore, the weight assigned to the total contrastive loss is gradually reduced. As the number of training rounds increases, the value of α decreases linearly from 1 to 0.

[0113] Experimental results on multiple high-dimensional gene expression datasets demonstrate that the knowledge-injected model constructed in this invention exhibits better robustness and predictive performance in predicting drug responses compared to traditional machine learning models (such as XGBoost, LASSO, Ridge, and PAC) and deep learning models (such as MLP, NODE, and Tab Transformer), improving the Pearson correlation coefficient by an average of 10.45%. Furthermore, by combining diverse sparse feature extraction and local context learning, the model can more effectively capture important information from the data while avoiding overfitting.

[0114] Example 2

[0115] To verify the effectiveness of the drug response prediction method for high-dimensional small sample data described in this invention, this embodiment mainly uses 6 datasets with a feature number d much greater than the sample number n and 4 datasets with a feature number d close to the sample number n, including non-small cell lung cancer (MNSCLC), cutaneous melanoma (CM), pancreatic ductal carcinoma (PDAC), breast cancer (BRCA), colorectal cancer (CRC), chondrosarcoma (CH), melanoma (ME), breast cancer (BC), small cell lung cancer (SCLC), and non-small cell lung cancer (NSCLC).

[0116] The experimental configuration for this embodiment is as follows:

[0117] The dataset was divided into training, validation, and test sets in a 6:2:2 ratio. The Pearson similarity coefficient (PCC) was used as the performance metric (calculating the Pearson similarity between predicted and actual values, ranging from -1 to 1; a value closer to 1 indicates better performance). To demonstrate the model's robustness, the data was randomly divided three times, and the model was trained three times according to the division results. The mean and standard deviation of the PCC performance on the test set were then calculated.

[0118] The experimental results are as follows:

[0119] Model performance evaluation on 6 datasets where the number of features d is much greater than the number of samples n: On 5 of the 6 datasets where the number of features d is much greater than the number of samples n, the proposed Knowledge Injection Model (KNDI) outperforms classical machine learning and tabular deep learning baselines, such as... Figure 2 As shown, for each dataset, the best-performing model is indicated in bold, and the second-best is indicated by underline. Among these, the present invention performs best on the MNSCLC, CM, PDCA, and CH datasets, with PCC improvements of approximately 8.09%, 15.17%, 25.25%, 22.94%, and 1.69% over the best baseline, respectively. PLATO performs best on the BRCA dataset, outperforming the present invention by 10.46% in PCC. Overall, the present invention achieves an average improvement of 10.45% in PCC compared to the best-performing baseline.

[0120] Model performance evaluation on datasets with 4 features (d) close to the number of samples (n): On datasets with 4 features (d) close to the number of samples (n), the knowledge injection model proposed in this invention outperforms classical machine learning and tabular deep learning baselines on 3 datasets, such as... Figure 2As shown. Specifically, this invention achieves state-of-the-art performance on the ME, BC, and NSCLC datasets, with PCC improvements of approximately 0.80%, 1.90%, and 0.12% respectively compared to the best baseline, although it performs 1.03% worse than PLATO on the SCLC dataset. Overall, this invention achieves an average improvement of 0.45% in PCC compared to the best-performing baseline.

[0121] Compared to datasets with a feature count d close to the sample size n, this invention exhibits a more significant performance improvement on datasets with a feature count d much larger than the sample size n. This demonstrates that the invention is robust to overfitting risks arising from high-dimensional and finite-sample scenarios. This improvement can be attributed to the integration of auxiliary domain knowledge, diversified sparse feature extraction, and contextual learning for local alignment.

[0122] In summary, the drug response prediction method for high-dimensional, small-sample data described in this invention constructs a knowledge injection model, utilizes feature embeddings extracted from a biomedical knowledge graph as additional input, and introduces a diversified sparse feature extraction module and a local alignment context learning submodule. The diversified sparse feature extraction module performs sparse feature selection and selective feature extraction on gene features and their embeddings, combined with contrastive loss constraints, to extract diverse high-level features from high-dimensional data, effectively reducing data dimensionality and extracting feature diversity and uniqueness. The local alignment context learning submodule extracts potential information interactions between high-level feature embeddings through a neighborhood attention mechanism and maintains the original high-level feature information through a self-alignment strategy. This invention, while avoiding overfitting, can more effectively capture important information in high-dimensional, small-sample data, significantly improving the accuracy of drug response prediction.

[0123] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0127] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for predicting drug response using high-dimensional, small-sample data, characterized in that, include: A knowledge injection model is constructed, including a data preprocessing module, a diversified sparse feature extraction module, a multi-branch context learning module, and a multi-branch feature fusion and prediction module; The multi-branch context learning module includes a local alignment context learning sub-module and inter-branch attention; The data preprocessing module is used to extract gene features and their embeddings from the genomic dataset; Each branch of the diversified sparse feature extraction module performs sparse feature selection on gene features and their embeddings, using the following formula: ； in, This represents the sparse feature selection of the i-th branch. k represents the total number of branches; This represents the sparse feature weight matrix of the i-th branch. Indicates gene feature embedding, Inputs selected for sparse features include gene features and gene feature embeddings; All branches are related to gene characteristics After performing sparse feature selection, output the sparse feature selected features. All branches embedding of gene features After performing sparse feature selection, the output is the feature embedding after sparse feature selection. ; Selective feature extraction is performed on the sparse feature selection features and their embeddings to obtain the high-level features and their embeddings for each branch, as shown in the formula: ； in, This represents the selective feature extraction of the i-th branch. This represents the attention weighting matrix for the i-th branch. This represents the selective feature weight matrix of the i-th branch. The input for selective feature extraction includes the features selected from sparse features and their embeddings; All branches select the sparse feature. After selective feature extraction, high-level features are output. Feature embeddings after sparse feature selection for all branches After selective feature extraction, the high-level feature embedding is output. ; The high-level feature embeddings of each branch are input into the local alignment context learning submodule. The neighborhood attention mechanism of the local alignment context learning submodule is used to extract the potential information interaction between the high-level feature embeddings, resulting in the feature embedding output by the neighborhood attention mechanism of each branch. The formula includes: ； ； ； in, For the high-level feature embedding of the i-th branch, For the QK operation of neighborhood attention on the i-th branch, For the V operation of neighborhood attention on the i-th branch, Indicates the first The index of a high-level feature embedding is the neighboring feature embedding. This indicates the number of features after selective feature extraction. This represents the m-th feature embedded in the high-level feature of the i-th branch. express neighborhood feature embedding, , and All are linear transformation weights for the i-th branch. This represents the bias of the m-th feature embedded in the high-level feature of the i-th branch; This represents the softmax activation function. This represents the feature embedding output by the neighborhood attention mechanism of the i-th branch; Then, the cross-attention mechanism of the local alignment context learning submodule is used to align the high-level feature embeddings of each branch with the feature embeddings output by the neighborhood attention mechanism, resulting in the target feature embedding for each branch. The formula includes: ； in, This represents the target feature embedding of the i-th branch. This represents the feature embedding output by the neighborhood attention mechanism of the i-th branch. For the high-level feature embedding of the i-th branch, , and Both represent the weights of cross-attention; The high-level features of each branch are input into the inter-branch attention, and the weight of each branch is calculated. The high-level features and target features of each branch are embedded and fused using the multi-branch feature fusion and prediction module. Then, the weights of each branch are used to sum the results to obtain the fused features. The fused features are then input into the multilayer perceptron to obtain the drug response prediction results.

2. The method for predicting drug response using high-dimensional small sample data according to claim 1, characterized in that, The data preprocessing module is used to extract gene features from publicly available genomic datasets. And utilize biomedical knowledge graphs to obtain gene characteristics. Embedded .

3. The method for predicting drug response using high-dimensional small sample data according to claim 1, characterized in that, The high-level features of each branch are input into the inter-branch attention, and the weight of each branch is calculated using the following formula: ； in, Indicates branch weight, Representing high-level features The initialized fixed input weight information, The weight matrix represents the attention between branches.

4. The method for predicting drug response using high-dimensional small sample data according to claim 1, characterized in that, The high-level features and target features of each branch are embedded and fused, and then the fused features are obtained by weighted summation using the weights of each branch. The formula is as follows: ； in, For fusion features, k represents the total number of branches. Let be the branch weight of the i-th branch. The initial weights represent the fused features. Embed the target features for the i-th branch. This represents the high-level feature of the i-th branch.

5. The method for predicting drug response using high-dimensional small sample data according to claim 1, characterized in that, The total loss for training the knowledge injection model includes mean squared error loss and total contrast loss.

6. The method for predicting drug response using high-dimensional small sample data according to claim 5, characterized in that, The formula for calculating the total contrast loss is as follows: ； in, The total contrast loss is N, where N is the number of pairwise combinations between branches, and S represents the set of pairwise combinations of high-level features from different branches. and These represent the high-level features of the i-th branch and the j-th branch, respectively. This represents the high-level features of the z-th sample in the i-th branch. This represents the high-level features of the z-th sample in the j-th branch; Let represent the contrast loss between the i-th branch and the j-th branch with respect to the z-th sample; The formula for calculating the contrast loss between two branches is: ； in, Indicates the first in the training batch One sample, Indicates temperature parameter, express and The similarity score between them.

Images