Drug response prediction method for high-dimensional small sample data

Abstract

The invention relates to the technical field of biological information, in particular to a drug response prediction method for high-dimensional small sample data, which comprises the following steps of: constructing a knowledge injection model, and embedding features extracted from a biomedical knowledge graph into the knowledge injection model as additional input, and a diversified sparse feature extraction module and a local alignment context learning sub-module are introduced. The diversified sparse feature extraction module extracts diversified high-level features from high-dimensional data by conducting sparse feature selection and selective feature extraction on gene features and embedding of the gene features, and the local alignment context learning sub-module extracts potential information interaction between high-level feature embedding through a neighborhood attention mechanism. And original high-level feature information is maintained through a self-alignment strategy. According to the method, important information in data can be more effectively captured while overfitting is avoided, and the accuracy of drug reaction prediction is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of bioinformatics, and in particular to a method for predicting drug reactions using high-dimensional small sample data. Background Art

[0002] Cancer is a complex genetic disease. Due to the heterogeneity of tumors, patients with the same type of cancer may experience different results when treated with the same drug. Cancer drug response prediction can help patients develop personalized treatment plans and plays a key role in precision medicine.

[0003] In recent years, with the development of high-throughput sequencing technology, the amount of data in the biomedical field has grown rapidly, especially in genomic research, which usually generates high-dimensional gene expression data. These data have tens of thousands of gene features, but the sample size is relatively small. This data characteristic poses a great challenge to machine learning models. Traditional statistical methods, such as Least Absolute Shrinkage and Selection Operator (LASSO) and Principal Component Analysis (PCA), although they perform well in dimensionality reduction and feature selection, are not effective in dealing with complex nonlinear patterns. On the other hand, decision trees and linear models such as XGBoost perform relatively stably on small sample data sets, but when dealing with high-dimensional data, these models are prone to overfitting and are difficult to accurately predict drug responses.

[0004] Neural networks perform well under conditions of abundant data, but their application in high-dimensional small sample scenarios is often limited by model complexity and insufficient data, which can easily lead to overfitting. In addition, although the biomedical field contains a wealth of auxiliary domain knowledge, this information has not been fully utilized in neural network models, resulting in the model's understanding of the data not being deep and comprehensive enough. Summary of the invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem of insufficient accuracy in drug response prediction for high-dimensional small sample data in the prior art.

[0006] In order to solve the above technical problems, the present invention provides a drug response prediction method for high-dimensional small sample data, comprising:

[0007] Construct a knowledge injection model, including a data preprocessing module, a diversified sparse feature extraction module, a multi-branch context learning module, and a multi-branch feature fusion and prediction module;

[0008] Extracting gene features and their embedding from genomic data sets using the data preprocessing module;

[0009] Using each branch of the diversified sparse feature extraction module to perform sparse feature selection and selective feature extraction on gene features and their embeddings, respectively, to obtain high-level features and their embeddings of each branch;

[0010] The multi-branch context learning module includes a local aligned context learning submodule and an inter-branch attention module; the high-level feature embedding of each branch is input into the local aligned context learning submodule, and the neighborhood attention mechanism of the local aligned context learning submodule is used to extract the potential information interaction between the high-level feature embeddings to obtain the feature embedding output by the neighborhood attention mechanism of each branch; the cross attention mechanism of the local aligned context learning submodule is then used to align the high-level feature embedding of each branch with the feature embedding output by the neighborhood attention mechanism to obtain the target feature embedding of each branch; the high-level features of each branch are input into the inter-branch attention to calculate the weight of each branch;

[0011] The multi-branch feature fusion and prediction module is used to embed and fuse the high-level features and target features of each branch, and then the weighted sum of each branch weight is used to obtain the fusion feature; the fusion feature is input into a multi-layer perceptron to obtain the drug reaction prediction result.

[0012] Preferably, the data preprocessing module is used to extract gene features x from a public genomic data set. o ; and use the biomedical knowledge graph to obtain gene features x o Embedded o .

[0013] Preferably, each branch of the diversified sparse feature extraction module is used to perform sparse feature selection on gene features and their embeddings respectively, and the formula is:

[0014]

[0015] Among them, SM (i) (·) represents the sparse feature selection of the i-th branch, i = 1, 2, …, k, k represents the total number of branches; represents the sparse feature weight matrix of the i-th branch, E o represents gene feature embedding, x 1 Inputs for sparse feature selection, including gene features and gene feature embeddings;

[0016] All branches have gene features x o After sparse feature selection, output the feature X after sparse feature selection s ; All branches embed gene features E o After sparse feature selection, the feature embedding E after sparse feature selection is outputs .

[0017] Preferably, selective feature extraction is performed on the features and their embeddings after sparse feature selection to obtain high-level features and their embeddings of each branch, and the formula is:

[0018]

[0019] Among them, FE (i) (·) represents the selective feature extraction of the i-th branch, represents the attention weight matrix of the i-th branch, represents the selective feature weight matrix of the i-th branch, x 2 The input for selective feature extraction includes the features after sparse feature selection and their embedding;

[0020] All branches have the feature X after sparse feature selection s After selective feature extraction, high-level features X are output e ; Feature embedding E after sparse feature selection for all branches s After selective feature extraction, the high-level feature embedding E is output e .

[0021] Preferably, the neighborhood attention mechanism of the local alignment context learning submodule is used to extract the potential information interaction between high-level feature embeddings, and the feature embedding output by the neighborhood attention mechanism of each branch is obtained. The formula includes:

[0022]

[0023] in, is the high-level feature embedding of the i-th branch, is the QK operation of neighborhood attention on the i-th branch, is the V operation of the neighborhood attention on the i-th branch, ρ(m) represents the index of the neighborhood feature embedding adjacent to the m-th high-level feature embedding, m represents the number of features after selective feature extraction, represents the mth feature of the high-level feature embedding of the i-th branch, express Neighborhood feature embedding, and are the linear transformation weights of the i-th branch, represents the bias of the mth feature embedded in the high-level feature of the i-th branch; σ represents the softmax activation function, Represents the feature embedding output by the neighborhood attention mechanism of the i-th branch.

[0024] Preferably, the cross attention mechanism of the local alignment context learning submodule is used to align the high-level feature embedding of each branch with the feature embedding output by the neighborhood attention mechanism to obtain the target feature embedding of each branch, and the formula is:

[0025]

[0026] in, represents the target feature embedding of the i-th branch, represents the feature embedding output by the neighborhood attention mechanism of the i-th branch, is the high-level feature embedding of the i-th branch, W q1 , W k1 and W v1 Both represent the weights of cross attention.

[0027] Preferably, the high-level features of each branch are input into the inter-branch attention, and the weight of each branch is calculated as follows:

[0028] w t =Softmax(init(X e ) a )

[0029] Among them, w t represents the branch weight, init(X e ) represents the high-level feature X e Initialized fixed input weight information, w a The weight matrix representing the attention between branches.

[0030] Preferably, the high-level features and target features of each branch are embedded and fused, and then the weighted sum of each branch weight is used to obtain the fused features, and the formula is:

[0031]

[0032] Among them, X h is the fusion feature, k represents the total number of branches, is the branch weight of the i-th branch, w 1 ,w 2 represents the initialization weight of the fusion feature, is the target feature embedding of the i-th branch, is the high-level feature of the i-th branch.

[0033] Preferably, the total loss of training the knowledge injection model includes mean square error loss and total contrast loss.

[0034] Preferably, the calculation formula of the total contrast loss is:

[0035]

[0036] Among them, L c is the total contrast loss, N is the number of combinations between branches, and S represents the set of pairwise combinations of high-level features of different branches; and Represent the high-level features of the i-th branch and the j-th branch respectively; represents the high-level features of the zth sample in the i-th branch, Represents the high-level features of the zth sample in the jth branch; represents the contrast loss between the i-th branch and the j-th branch regarding the z-th sample;

[0037] The formula for calculating the contrast loss between the two branches is:

[0038]

[0039] Where s represents the sth sample in the training batch, τ represents the temperature parameter, express and The similarity score between .

[0040] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0041] The present invention discloses a method for predicting drug response of high-dimensional small sample data. By constructing a knowledge injection model, the feature embedding extracted from the biomedical knowledge graph is used as an additional input, and a diversified sparse feature extraction module and a local alignment context learning submodule are introduced. The diversified sparse feature extraction module extracts diversified high-level features from high-dimensional data by performing sparse feature selection and selective feature extraction on gene features and their embeddings, and combines contrast loss constraints to effectively reduce the data dimension and extract the diversity and uniqueness of features. The local alignment context learning submodule extracts the potential information interaction between high-level feature embeddings through the neighborhood attention mechanism, and maintains the original high-level feature information through the self-alignment strategy. While avoiding overfitting, the present invention can more effectively capture important information in high-dimensional small sample data, significantly improving the accuracy of drug response prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0043] Figure 1 is a flowchart of a method for predicting drug response based on high-dimensional small sample data of the present invention, wherein Figure 1 The a in is the data preprocessing module. Figure 1 The b in is a diversified sparse feature extraction module. Figure 1 The c in is a multi-branch context learning module. Figure 1 The d in is the multi-branch feature fusion and prediction module;

[0044] Figure 2 This is the performance comparison result of the present invention and the existing method on multiple data sets in Example 2. DETAILED DESCRIPTION

[0045] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0046] Embodiment 1

[0047] Reference Figure 1 As shown, the present invention provides a drug response prediction method for high-dimensional small sample data, comprising:

[0048] Constructing a knowledge injection model, including a data preprocessing module, a diversified sparse feature extraction module, a multi-branch context learning module and a multi-branch feature fusion and prediction module; the multi-branch context learning module includes a local alignment context learning submodule and inter-branch attention;

[0049] Extracting gene features and their embedding from genomic data sets using the data preprocessing module;

[0050] Using each branch of the diversified sparse feature extraction module to perform sparse feature selection and selective feature extraction on gene features and their embeddings, respectively, to obtain high-level features and their embeddings of each branch;

[0051] The high-level feature embedding of each branch is input into the local aligned context learning submodule, and the neighborhood attention mechanism of the local aligned context learning submodule is used to extract the potential information interaction between the high-level feature embeddings to obtain the feature embedding output by the neighborhood attention mechanism of each branch; the cross attention mechanism of the local aligned context learning submodule is then used to align the high-level feature embedding of each branch with the feature embedding output by the neighborhood attention mechanism to obtain the target feature embedding of each branch; the high-level features of each branch are input into the inter-branch attention to calculate the weight of each branch;

[0052] The multi-branch feature fusion and prediction module is used to embed and fuse the high-level features and target features of each branch, and then the weighted sum of each branch weight is used to obtain the fusion feature; the fusion feature is input into a multi-layer perceptron to obtain the drug reaction prediction result.

[0053] The following is a detailed introduction to the processing of each module in the knowledge injection model.

[0054] The data preprocessing module extracts gene features x from the public genomic dataset o ∈R n×d , where n represents the number of samples and d represents the number of features; and the gene feature x is obtained by using the biomedical knowledge graph (KG) o Embedded o ∈R d×c , where c represents the dimension of feature embedding.

[0055] In the feature extraction and context learning process of the model, the present invention uses feature embedding extracted from the biomedical knowledge graph (such as the association information of genes, diseases, and drugs) as additional input. This knowledge injection mechanism effectively enhances the model's ability to understand gene features and improves the effect of drug response prediction.

[0056] The gene features and their embeddings are input into the diversified sparse feature extraction module.

[0057] The Diversified Sparse Feature Extraction module selects important features from high-dimensional gene expression data by introducing sparse mechanisms and contrast constraints. Specifically, the module adopts a multi-branch strategy, where each branch is responsible for abstracting features from different angles, and the contrast loss function is used to ensure that the feature extraction of each branch has sufficient differences. This diverse sparse feature extraction mechanism effectively reduces the dimensionality of the model input and retains the most important features.

[0058] In order to efficiently extract effective feature information from high-dimensional gene expression data, the diversified sparse feature extraction module first uses a multi-branch strategy to perform sparse feature selection on gene features and their embeddings. The formula for sparse feature selection is as follows:

[0059]

[0060] Among them, SM (i) (·) represents the sparse feature selection of the i-th branch, i = 1, 2, …, k, k represents the total number of branches; represents the sparse feature weight matrix of the i-th branch, E o represents gene feature embedding, x 1 Inputs for sparse feature selection include gene features and gene feature embeddings.

[0061] Sparse feature weight matrix It is composed of multiple layers of linear networks, and each layer of linear networks must perform LeakReLU activation function, batch normalization and dropout operations in sequence. The LeakReLU activation function is used to maintain network sparsity, and the dropout operation is used to randomly discard neurons.

[0062] All branches have gene features x o After sparse feature selection, output the feature X after sparse feature selection s ∈R k ×d×n ; All branches embed gene features E o After sparse feature selection, the feature embedding E after sparse feature selection is output s ∈R k×d×c , expressed as:

[0063] [X s ,E s ]=stack[SM (1) (T o ),SM (2) (T o ),…,SM (k) (T o )]

[0064] Among them, T o ∈R d×(n+c) Represents gene feature x 0 and its embedding E 0 combination.

[0065] Then select the feature X after sparse feature selection s and its embedding E s Perform selective feature extraction. In order to effectively extract high-level features, a learnable weight matrix w is used e1 ∈R k×m×d To capture the collaborative interaction between the selected features and perform dimensionality reduction, m represents the number of extracted features. At the same time, the selectively extracted features and their embeddings are weighted by the attention mechanism to focus on significant and effective high-level features. The attention weighting matrix of the attention mechanism is w e0 ∈R k×m×d .

[0066] The formula for selective feature extraction is:

[0067]

[0068] Among them, FE (i) (·) represents the selective feature extraction of the i-th branch, represents the attention weight matrix of the i-th branch, represents the selective feature weight matrix of the i-th branch, x 2 It is the input for selective feature extraction, including the features after sparse feature selection and their embedding.

[0069] All branches have the feature X after sparse feature selection s After selective feature extraction, high-level features X are output e ∈R k×m×n ; Feature embedding E after sparse feature selection for all branches s After selective feature extraction, the high-level feature embedding E is output e ∈R k×m×c , expressed as:

[0070]

[0071] Among them, T s ∈R k×d×(n+c) Represents the feature X after sparse feature selection s and its embedding E s combination.

[0072] To ensure that the high-level features extracted by different branches have diversity, when training the knowledge injection model, contrast loss constraints are used between the high-level features output by multiple branches of the diversified sparse feature extraction module. The specific steps are as follows:

[0073] Calculate the similarity scores between high-level features output by different branches:

[0074]

[0075] in, represents the high-level features of the zth sample in the i-th branch, represents the high-level features of the zth sample of the jth branch, sim represents the similarity between features of different branches calculated by vector multiplication, express and The similarity score between .

[0076] The contrast loss is calculated between every two branches, and the corresponding sample similarity scores between branches are regarded as positive sample pairs, while the remaining samples are negative sample pairs.

[0077] The formula for calculating the contrast loss between the two branches is:

[0078]

[0079] in, represents the contrastive loss between the i-th branch and the j-th branch with respect to the z-th sample, b represents the total number of samples in the training batch, s represents the s-th sample in the training batch, and τ represents the temperature parameter.

[0080] Combine the contrast losses of all branch combinations to calculate the total contrast loss, the formula is:

[0081]

[0082] Among them, L c is the total contrast loss, N is the number of combinations between branches, and S represents the set of high-level features of different branches that are combined with each other. and Represent the high-level features of the i-th branch and the j-th branch respectively.

[0083] During the training process, the total contrast loss is used to optimize the weight matrix of each branch, so that after training, the high-level features extracted by each branch have diverse characteristics.

[0084] The diversified sparse feature extraction module adopts a multi-branch strategy and combines diversified sparse feature extraction, selective feature extraction and contrast loss constraint mechanism to extract high-level features from high-dimensional gene expression data. Diversified sparse feature extraction uses frozen feature embedding, weight network and sigmoid activation function to assign weight scores to gene features and their embeddings of each branch to achieve the purpose of feature selection. Selective feature extraction extracts features and their embeddings after sparse feature selection through two linear networks. The output of one network uses sigmoid activation function to weight the features extracted by the other network, so as to achieve the purpose of selective feature extraction and obtain high-level features and their embeddings of multiple branches. Contrast loss constraint first calculates the similarity of high-level features between different branches, and then differentiates the similarity between the high-level features extracted between each branch through total contrast loss, so as to achieve the purpose of diversified high-level features and their embeddings.

[0085] The high-level features and their embeddings are input into the multi-branch context learning module, where the high-level feature embeddings are input into the local alignment context learning submodule, and the high-level features are input into the inter-branch attention.

[0086] In order to further improve the model's understanding of the relationship between features, the present invention proposes a local alignment context learning submodule. This module uses a neighborhood attention mechanism to capture the contextual relationship between high-dimensional features, and uses a self-alignment mechanism to ensure that the original information of the features is retained during the learning process. The local alignment context learning submodule improves the accuracy of feature representation by effectively fusing local information with contextual relationships.

[0087] The local alignment context learning submodule processes high-level feature embedding as follows:

[0088] Due to the small sample size, excessive contextual learning may introduce bias and cause overfitting. Therefore, the present invention calculates the comprehensive neighborhood information of the high-level feature embedding of each branch based on the neighborhood attention mechanism (NA) mechanism to extract the potential interactions between the high-level feature embeddings.

[0089] The high-level feature embedding of each branch is input into the neighborhood attention mechanism, and the neighborhood attention mechanism is used to extract the potential information interaction between the high-level feature embeddings to obtain the feature embedding output by the neighborhood attention mechanism of each branch. The formula includes:

[0090]

[0091] in, is the high-level feature embedding of the i-th branch, is the QK operation of neighborhood attention on the i-th branch, is the V operation of the neighborhood attention on the i-th branch, ρ(m)∈R l represents the index of the embedding of the adjacent l neighborhood features of the mth high-level feature, m represents the number of features after selective feature extraction, represents the mth feature of the high-level feature embedding of the i-th branch, express The l neighborhood feature embeddings of and are the linear transformation weights of the i-th branch, represents the bias of the mth feature embedded in the high-level feature of the i-th branch; σ represents the softmax activation function, represents the feature embedding output by the neighborhood attention mechanism of the i-th branch, and c represents the dimension of the feature embedding.

[0092] Furthermore, in order to prevent the misalignment problem between feature input and feature embedding that may occur in the subsequent multi-branch feature fusion, the present invention also uses a self-alignment mechanism in the local alignment context learning submodule. The self-alignment mechanism preserves the original information as much as possible by aligning the potential interactions between the extracted high-level feature embeddings to the original high-level feature embeddings through a cross-attention mechanism, thereby ensuring the information consistency between the high-level features and their embeddings.

[0093] The input and output of the neighborhood attention mechanism of each branch are aligned through the self-alignment mechanism, that is, the high-level feature embedding and the feature embedding output by the neighborhood attention mechanism, to obtain the target feature embedding of each branch. The formula is as follows:

[0094]

[0095] in, represents the target feature embedding of the i-th branch, W q1 , W k1 and W v1 Both represent the weights of cross attention.

[0096] The local alignment context learning submodule is able to capture detailed contextual information while preserving the original feature embeddings.

[0097] The process of processing high-level features by inter-branch attention is as follows:

[0098] Since the local aligned context learning submodule uses multiple branches to extract high-level features and learn the context information embedded in multiple branch features, the present invention focuses on the importance of each branch, converts the high-level features into a multi-layer linear network, and normalizes the output results of the multi-layer linear network to between 0 and 1 through a softmax activation function. At the same time, the normalized results are added to 1, thereby achieving the purpose of learning the importance of each branch.

[0099] The calculation formula for each branch weight is as follows:

[0100] w t =Softmax(init(X e ) a )

[0101] Among them, w t ∈R k represents the branch weight, init(X e )∈R k×m Represents high-level features X e Initialized fixed input weight information; w a ∈R m×1 The weight matrix representing the attention between branches is composed of multiple layers of linear networks. Each layer of linear network performs LeakReLU activation function, batch normalization and dropout operations in turn.

[0102] In the final stage of the knowledge injection model, the features of different branches are weighted and added together for fusion. The fused features are processed through a multi-layer perceptron network and the final output is the prediction result of drug response.

[0103] In the multi-branch feature fusion and prediction module, the high-level features of each branch and its target features are embedded and fused element-by-element multiplication, and then weighted summed using the branch weights to obtain the fused features. The formula is:

[0104]

[0105] Among them, X h ∈R n×h is the fusion feature, k represents the total number of branches, is the branch weight of the i-th branch, w 1 ,w 2 ∈R c×h represents the initialization weight of the fusion feature, is the target feature embedding of the i-th branch, is the high-level feature of the i-th branch.

[0106] Input the fusion features into the multi-layer perceptron to obtain the drug response prediction y∈R n .

[0107] During the training process, the knowledge injection model uses the mean square error (MSE) loss to minimize the error between the predicted value and the true value, as follows:

[0108]

[0109] Among them, L r is the mean square error loss, b represents the total number of samples in the training batch, and y z and They represent the predicted value and true value of drug response of the zth sample respectively.

[0110] The total loss of the knowledge injection model also uses contrastive loss to minimize the similarity between branches, as follows:

[0111] L=L r +αL c

[0112] Among them, L is the total loss, L r is the mean square error loss, L c is the total contrast loss, and α is the contrast loss weight. The value range of α is [0,1]. Since the difference between branches becomes more significant during the training process, more attention is paid to the difference between the predicted value and the true value, that is, the mean square error loss. Therefore, the weight assigned to the total contrast loss is gradually reduced. As the number of training rounds increases, the value of α decreases linearly from 1 to 0.

[0113] Experimental results on multiple high-dimensional gene expression data sets show that the knowledge injection model constructed by the present invention shows better robustness and prediction performance when predicting drug response than traditional machine learning models (such as XGBoost, LASSO, Ridge and PAC, etc.) and deep learning models (such as MLP, NODE and Tab Transformer, etc.), and improves the Pearson correlation coefficient by an average of 10.45%. In addition, by combining diverse sparse feature extraction and local context learning, the model can more effectively capture important information in the data while avoiding overfitting.

[0114] Embodiment 2

[0115] In order to verify the effectiveness of the drug response prediction method for high-dimensional small sample data described in the present invention, this embodiment mainly uses 6 data sets whose feature number d is much larger than the sample number n and 4 data sets whose feature number d is close to the sample number n, including non-small cell lung cancer (MNSCLC), cutaneous melanoma (CM), pancreatic ductal carcinoma (PDAC), breast cancer (BRCA), colorectal cancer (CRC), chondrosarcoma (CH), melanoma (ME), breast cancer (BC), small cell lung cancer (SCLC) and non-small cell lung cancer (NSCLC).

[0116] The experimental configuration of this embodiment is as follows:

[0117] The dataset is divided into training set, validation set and test set according to the ratio of 6:2:2. The Pearson Similarity Coefficient (PCC) is used as the measurement indicator (the Pearson similarity value between the predicted value and the actual value is calculated, ranging from -1 to 1, and the closer to 1, the better the performance). In order to reflect the robustness of the model, the data is randomly divided 3 times, and the model is trained 3 times according to the division results, and the average and standard deviation of the performance (PCC) on the test set are calculated.

[0118] The experimental results are as follows:

[0119] Model performance evaluation on 6 datasets where the number of features d is much larger than the number of samples n: On 6 datasets where the number of features d is much larger than the number of samples n, the knowledge injection model (KNDI) proposed in this paper outperforms the classical machine learning and tabular deep learning baselines on 5 datasets, such as Figure 2As shown, for each data set, the best performing model is indicated in bold, and the second best is indicated by underline. Among them, the present invention has the best performance on the MNSCLC, CM, PDCA and CH data sets, and PCC is improved by about 8.09%, 15.17%, 25.25%, 22.94% and 1.69% respectively over the best baseline. PLATO performs best on the BRCA data set, and is 10.46% better than the present invention in terms of PCC. Overall, compared with the best performing baseline, the present invention has an average improvement of 10.45% in terms of PCC.

[0120] Model performance evaluation on 4 datasets with the number of features d close to the number of samples n: In the 4 datasets with the number of features d close to the number of samples n, the performance of the knowledge injection model proposed in this paper is better than the classical machine learning and tabular deep learning baselines on 3 datasets, such as Figure 2 Specifically, the present invention achieves the best performance on the ME, BC, and NSCLC datasets, with PCC improvements of approximately 0.80%, 1.90%, and 0.12%, respectively, compared to the best baseline, although it performs 1.03% lower than PLATO on the SCLC dataset. Overall, the present invention achieves an average improvement of 0.45% in PCC compared to the best performing baseline.

[0121] Compared with the dataset with the number of features d close to the number of samples n, the present invention shows more significant performance improvement gains on the dataset with the number of features d much larger than the number of samples n, which shows that the present invention is robust under the risk of overfitting caused by high-dimensional and limited sample scenarios. This improvement can be attributed to the integration of auxiliary domain knowledge, diversified sparse feature extraction, and local aligned context learning.

[0122] In summary, the present invention discloses a method for drug response prediction for high-dimensional small sample data. By constructing a knowledge injection model, the feature embedding extracted from the biomedical knowledge graph is used as an additional input, and a diversified sparse feature extraction module and a local alignment context learning submodule are introduced. The diversified sparse feature extraction module extracts diversified high-level features from high-dimensional data by performing sparse feature selection and selective feature extraction on gene features and their embeddings, and combines contrast loss constraints to effectively reduce the data dimension and extract the diversity and uniqueness of features. The local alignment context learning submodule extracts the potential information interaction between high-level feature embeddings through the neighborhood attention mechanism, and maintains the original high-level feature information through the self-alignment strategy. While avoiding overfitting, the present invention can more effectively capture important information in high-dimensional small sample data, significantly improving the accuracy of drug response prediction.

[0123] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0124] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0125] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0127] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A drug response prediction method for high-dimensional small sample data, characterized in that: include: Construct a knowledge injection model, including a data preprocessing module, a diversified sparse feature extraction module, a multi-branch context learning module, and a multi-branch feature fusion and prediction module; The multi-branch context learning module includes a local alignment context learning submodule and inter-branch attention; Extracting gene features and their embedding from genomic data sets using the data preprocessing module; Using each branch of the diversified sparse feature extraction module to perform sparse feature selection and selective feature extraction on gene features and their embeddings, respectively, to obtain high-level features and their embeddings of each branch; The high-level feature embedding of each branch is input into the local aligned context learning submodule, and the neighborhood attention mechanism of the local aligned context learning submodule is used to extract the potential information interaction between the high-level feature embeddings to obtain the feature embedding output by the neighborhood attention mechanism of each branch; the cross attention mechanism of the local aligned context learning submodule is then used to align the high-level feature embedding of each branch with the feature embedding output by the neighborhood attention mechanism to obtain the target feature embedding of each branch; the high-level features of each branch are input into the inter-branch attention to calculate the weight of each branch; The multi-branch feature fusion and prediction module is used to embed and fuse the high-level features and target features of each branch, and then the weighted sum of each branch weight is used to obtain the fusion feature; The fused features are input into a multi-layer perceptron to obtain the drug response prediction results.

2. The drug response prediction method for high-dimensional small sample data according to claim 1, characterized in that: The data preprocessing module is used to extract gene features x from the public genome dataset o ; and use the biomedical knowledge graph to obtain gene features x o Embedded o .

3. The drug response prediction method for high-dimensional small sample data according to claim 2, characterized in that: Each branch of the diversified sparse feature extraction module is used to perform sparse feature selection on gene features and their embeddings, and the formula is: Among them, SM (i) (·) represents the sparse feature selection of the i-th branch, i = 1, 2, …, k, k represents the total number of branches; represents the sparse feature weight matrix of the i-th branch, E o represents gene feature embedding, x1 is the input of sparse feature selection, including gene features and gene feature embedding; All branches have gene features x o After sparse feature selection, output the feature X after sparse feature selection s ; All branches embed gene features E o After sparse feature selection, the feature embedding E after sparse feature selection is output.

4. The drug response prediction method for high-dimensional small sample data according to claim 3, characterized in that: Selective feature extraction is performed on the features and their embeddings after sparse feature selection to obtain the high-level features and their embeddings of each branch. The formula is: Among them, FE (i) (·) represents the selective feature extraction of the i-th branch, represents the attention weight matrix of the i-th branch, represents the selective feature weight matrix of the i-th branch, x2 is the input of selective feature extraction, including the features after sparse feature selection and their embedding; All branches have the feature X after sparse feature selection s After selective feature extraction, high-level features X are output e ; Feature embedding E after sparse feature selection for all branches s After selective feature extraction, the high-level feature embedding E is output e .

5. The drug response prediction method for high-dimensional small sample data according to claim 1, characterized in that: The neighborhood attention mechanism of the local alignment context learning submodule is used to extract the potential information interaction between high-level feature embeddings, and the feature embedding output by the neighborhood attention mechanism of each branch is obtained. The formula includes: in, is the high-level feature embedding of the i-th branch, is the QK operation of neighborhood attention on the i-th branch, is the V operation of the neighborhood attention on the i-th branch, ρ(m) represents the index of the neighborhood feature embedding adjacent to the m-th high-level feature embedding, m represents the number of features after selective feature extraction, represents the mth feature of the high-level feature embedding of the i-th branch, express Neighborhood feature embedding, and are the linear transformation weights of the i-th branch, represents the bias of the mth feature embedded in the high-level feature of the i-th branch; σ represents the softmax activation function, Represents the feature embedding output by the neighborhood attention mechanism of the i-th branch.

6. The method for predicting drug response based on high-dimensional small sample data according to claim 5, characterized in that: The cross-attention mechanism of the local alignment context learning submodule is used to align the high-level feature embedding of each branch with the feature embedding output by the neighborhood attention mechanism to obtain the target feature embedding of each branch. The formula is: in, represents the target feature embedding of the i-th branch, represents the feature embedding output by the neighborhood attention mechanism of the i-th branch, is the high-level feature embedding of the i-th branch, W q1 , W k1 and W v1 Both represent the weights of cross attention.

7. The drug response prediction method for high-dimensional small sample data according to claim 1, characterized in that: The high-level features of each branch are input into the inter-branch attention, and the weight of each branch is calculated. The formula is: w t =Softmax(init(X e )w a ) Among them, w t represents the branch weight, init(X e ) represents the high-level feature X e Initialized fixed input weight information, w a The weight matrix representing the attention between branches.

8. The method for predicting drug response based on high-dimensional small sample data according to claim 1, characterized in that: The high-level features and target features of each branch are embedded and fused, and then the weighted sum of each branch weight is used to obtain the fusion feature. The formula is: Among them, X h is the fusion feature, k represents the total number of branches, is the branch weight of the i-th branch, w1, w2 represent the initialization weights of the fusion features, is the target feature embedding of the i-th branch, is the high-level feature of the i-th branch.

9. The drug response prediction method for high-dimensional small sample data according to claim 1, characterized in that: The total loss for training the knowledge injection model includes mean square error loss and total contrast loss.

10. The method for predicting drug response based on high-dimensional small sample data according to claim 9, characterized in that: The calculation formula of the total contrast loss is: Among them, L c is the total contrast loss, N is the number of combinations between branches, and S represents the set of pairwise combinations of high-level features of different branches; and Represent the high-level features of the i-th branch and the j-th branch respectively; represents the high-level features of the zth sample in the i-th branch, Represents the high-level features of the zth sample in the jth branch; represents the contrast loss between the i-th branch and the j-th branch regarding the z-th sample; The formula for calculating the contrast loss between the two branches is: Where s represents the sth sample in the training batch, τ represents the temperature parameter, express and The similarity score between .

Images