Multimodal and phenotype association mining method based on interpretable autoencoder

Abstract

The application discloses a multi-omics and phenotype association mining method based on an interpretable autoencoder. First, a priori knowledge residual sparse encoder is built, high-dimensional multi-omics features are encoded into low-dimensional potential expressions containing all omics hidden associations by using priori knowledge. Second, a decoder is constructed to regenerate high-dimensional features from low-dimensional potential expressions, and a loss function is established according to the difference between the high-dimensional features and the original multi-omics features. Then, an explanatory analysis module based on additive feature attribution is added to establish a complete interpretable residual variational sparse autoencoder. Third, a residual neural network is established with low-dimensional expressions as input to complete downstream classification or regression tasks. Finally, the interpretable residual variational sparse autoencoder and the residual neural network are trained respectively, and the entire model framework after training can realize multi-omics and phenotype association mining, the input of which is multi-omics data and priori knowledge, and the output of which is neuron explanatory score and the result of downstream classification or regression tasks.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of biological information, and particularly relates to a multi-omics and phenotype association mining method based on an interpretable autoencoder. BACKGROUND

[0002] With the reduction in cost and improvement in efficiency of high-throughput sequencing technology in recent years, biological scientists have produced a large amount of human omics data of different types, which makes various applications based on multi-omics analysis possible. Compared with traditional analysis methods based on a single omics, multi-omics analysis can accurately analyze the operation of the human physiological system from a deeper perspective. In addition, in recent years, deep learning algorithms have become one of the most promising methods in multi-omics data analysis due to their prediction performance and ability to capture nonlinear and hierarchical features. Therefore, multi-omics analysis methods based on deep learning have become an important branch in the field of biological information research in recent years, and the exploration of omics and human phenotype association patterns is a hot spot in the field.

[0003] Phenotype generally refers to the phenotype, which is the structural and functional characteristics of an organism that can be observed, such as morphological and behavioral characteristics, disease subtypes, and drug responses. In biomedical research, combining multi-omics can analyze phenotypes at the molecular level, and can achieve the mining of phenotype-related biomarkers, the exploration of human microscopic regulation mechanisms, the discovery of new targeted drugs, and the development of precision medical solutions, etc. Therefore, a large number of scholars are currently engaged in deep learning research on multi-omics phenotype association, and have made a lot of progress. For example, Zhang et al. proposed a deep latent space fusion (DLSF) method to learn a consistent manifold in the sample latent space, integrate multi-omics data for disease subtype identification; Sun et al. proposed a multimodal deep neural network (MDNNMD) that integrates multi-dimensional data, which integrates copy number variation, gene expression and clinical data, uses a fully connected neural network to calculate each omics score and integrates it in the later stage, and finally realizes the diagnosis of breast cancer subtypes; Arya et al. proposed a closed attention deep learning model based on a random forest classifier, which uses multi-modal data to generate information features to enhance the performance of breast cancer prognosis prediction.

[0004] However, most of the current multi-omics and phenotype association studies based on deep learning represented by the above method lack explainability in the deep learning part, so some statistical analysis mechanisms are usually added outside the deep learning module to achieve biological pathway or gene level explanation. Although the effectiveness of these statistical analysis mechanisms has been proven many times in past studies, this method is mostly completely separated from the deep learning module, that is, the most important deep learning part in the study is still a black box, which only serves as a predictor and cannot explain the prediction results from the perspective of the model. SUMMARY

[0005] In order to solve the above-mentioned problems, the present application proposes a multi-omics and phenotype association mining method based on an interpretable autoencoder. The interpretable autoencoder can also be called an interpretable residual variational sparse autoencoder, which consists of a prior knowledge residual variational sparse encoder, a decoder and an interpretability analysis module. The present application first builds a prior knowledge residual sparse encoder, which can use prior knowledge to encode high-dimensional multi-omics features into low-dimensional latent representations that contain all the hidden associations of the omics. Secondly, a decoder is constructed to regenerate high-dimensional features from the low-dimensional latent representations, and a loss function is established according to the difference between the high-dimensional features and the original multi-omics features to lay the foundation for the training of the entire autoencoder. Then, based on the above, an interpretability analysis module based on additive feature attribution is added to establish a complete interpretable residual variational sparse autoencoder. Thirdly, a residual neural network is established with the low-dimensional representation as the input to complete the downstream classification or regression task. Finally, the interpretable residual variational sparse autoencoder and the residual neural network are trained respectively, and the entire model framework after training can realize multi-omics and phenotype association mining, with multi-omics data and prior knowledge as input and neuron interpretability scores and the results of the downstream classification or regression task as output.

[0006] In order to achieve the above purpose, the specific technical solutions of the present application are as follows:

[0007] A multi-omics and phenotype association mining method based on an interpretable autoencoder, the steps are as follows:

[0008] Step (1) constructing a prior knowledge residual variational sparse encoder

[0009] Multi-omics high-dimensional data is redundant and full of noise, and needs to be preprocessed and reduced in dimension to reliably perform phenotype and omics association mining. The present application uses a combination of prior knowledge and sparse coding to perform variational coding of multi-omics high-dimensional data, and adds a residual connection to improve the training difficulty of the entire autoencoder model. The input of the prior knowledge residual variational sparse encoder is multi-omics data and a prior knowledge mask matrix, and the output is the encoding of the multi-omics data. Specifically as follows:

[0010] Firstly, the prior knowledge masking matrix is based on the full connection network encoder to build the sparse encoder structure. In the connection between the upper layer and the next layer of the encoder, the traditional autoencoder adopts the full connection form to connect, as follows:

[0011] y = f(W T x + b) (1)

[0012] Wherein y is the output vector of the next layer, x is the input vector of the upper layer, W is the weight matrix of the neural network layer, b is the bias vector, and f is the activation function. The present application introduces a prior knowledge masking matrix M on the basis of formula (1). M is a sparse matrix, and the elements in M take values of 0 or 1. In the signal pathway with biological prior knowledge, the value of 0 means that the signal pathway has not been opened, and there is no universal contact between them. The value of 1 means that the existing knowledge pathway is connected. After introducing M, the present application improves formula (1) to the following formula:

[0013] y = f((M * W) T x + b) (2)

[0014] After the processing of formula (2), the full connection encoder is improved to the prior knowledge sparse encoder in the present application.

[0015] Secondly, the hidden space probability distribution mechanism is added on the basis of the above to build the prior knowledge variational sparse encoder. The encoder is different from the traditional full connection encoder, which first learns the mean vector μ and the variance vector σ, and then generates the low-dimensional latent expression z by combining the mean and variance with the Gaussian distribution, as follows:

[0016] z = u + σ∈ (3)

[0017] ∈ is a random vector sampled from the Gaussian distribution N(0, I), and I is the unit matrix.

[0018] Finally, the multiple residual connection mechanism is added on the basis of the above to construct the prior knowledge residual variational sparse encoder. Assuming that the encoder has n layers, the present application adds a residual connection mechanism between the total input to the first layer output, the first layer output to the second layer output, and so on to the n-1 layer output. For each residual connection, its principle is based on the following formula:

[0019]

[0020] The output of the residual module is represented by x l The input of the residual module is represented by x l W s is an identity mapping, and W sF is a residual mapping that needs to be learned.

[0021] The prior knowledge residual variational sparse encoder is established through the above process.

[0022] Step (2) constructs a decoder and a prior knowledge residual variational sparse autoencoder.

[0023] The decoder input is the encoding of the multi-omics data output by step (1), and the output is the reconstructed feature x out The decoder and the encoder established in step (1) are coupled to construct a prior knowledge residual variational sparse autoencoder, which outputs low-dimensional latent representation of multi-omics after training, and lays a foundation for subsequent multi-omics and phenotype association decision-making. Specifically as follows:

[0024] First, a fully connected decoder is established. The decoder is based on the fully connected network of formula (1), and the number of layers and nodes of each layer is the same as that of the encoder, but the connection between nodes does not use the masking matrix, and there is no residual connection mechanism between layers.

[0025] Second, the decoder is coupled with the encoder of step (1) to construct a prior knowledge residual variational sparse autoencoder. Finally, the autoencoder is trained. The loss function of the training is as follows:

[0026] Loss=||x in -x out || 2 +KL[N(u,σ),N(0,I)] (5)

[0027]

[0028] x in is the input of the autoencoder, i.e. multi-omics data, KL is the divergence function, p(x) and q(x) are two different arbitrary polynomials, and N is a Gaussian function. The autoencoder is trained based on formulas (5) and (6). The training is performed together with the interpreter of step (3), and the explainability of the autoencoder deep learning model is realized accordingly. In addition, the multi-omics low-dimensional latent representation after training can be used as input for multi-omics and phenotype classification decision-making in step (4).

[0029] Step (3) constructs an interpreter based on additive feature attribution

[0030] Let f 原 be the model to be explained (autoencoder), g be the explanation model, focus on a local feature x 局 , g uses a simplified feature x 局 ′ through a mapping function so that x局 =h(x 局 ′), thus ensuring g(x) 局 ′)≈f 原 (h(x 局 Additive eigenvalues ​​can be attributed to a linear function of two variables, as shown in the following equation:

[0031]

[0032] Where M represents the number of simplified features, α i The effect value representing a characteristic is obtained by summing all characteristic effect values. 局 ′), so that g(x) 局 ′) Approaching f 原 (h(x 局 ′)) can be interpreted by explaining the model to understand f 原 Each feature in the equation is interpreted. In other words, the goal of an interpreter based on additive feature attribution is to construct an interpreter that is sufficiently close to f. 原 The importance of the feature is explained by the feature effect value α in formula (7), that is, calculating α is the ultimate goal of the interpreter.

[0033] This invention calculates α by minimizing the objective function ω, which is as follows:

[0034]

[0035] in Let represent the kernel function, τ(g) be the regularization term, and L be the squared loss function. The above equation means that g with respect to f... 原 The approximation is achieved by simplifying the loss L of a set of samples weighted by a kernel function in the input space. After minimizing ω, g and f 原 Fitting was achieved, and through The characteristic effect value α is calculated, thereby enabling the interpretation of each characteristic.

[0036] Step (4) Construct the residual network

[0037] The input to the residual network is the multi-omics low-dimensional latent expression trained in step (2). The output is the phenotypic classification decision result. The residual network used in this invention is a one-dimensional residual network, which consists of multiple residual blocks with different input and output dimensions. More specifically, each residual block consists of two convolutional blocks connected sequentially and residual connections. The source of the residual connection is the input of the first convolutional block, and the purpose is the output of the second convolutional block. The connection method is based on formula (4). In addition, pooling layers, four residual network modules, a global average pooling layer, and a fully connected layer are added to form the residual network. This residual network can achieve phenotypic classification decision after training.

[0038] In summary, the input of the present application is the prior knowledge masking matrix and the multi-omics data, and the output is the explanatory and phenotypic classification decision result at the level of omics features, which realizes the explainable association mining of multi-omics and phenotype.

[0039] The present application has the advantages of improving the accuracy, reliability and explainability of omics-phenotype association mining. On the one hand, the prior knowledge residual variational sparse autoencoder integrates multiple omics data and prior knowledge, and improves the accuracy and noise resistance of phenotypic classification decision through multi-omics integration based on prior knowledge and the proposed self-properties of the autoencoder. On the other hand, the effect value of the feature is calculated by introducing the interpreter, which realizes the feature-level explainability of the classification decision result of the overall deep learning model, avoids the "black box" problem of traditional deep learning, and improves the reliability of the model from the explanation level. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is the overall framework diagram of the present application.

[0041] Figure 2 is the construction diagram of the explainable residual variational autoencoder of the present application.

[0042] Figure 3 is the workflow diagram of the present application. DETAILED DESCRIPTION

[0043] The specific embodiments of the present application are further described below in combination with the drawings and technical solutions.

[0044] The overall framework of the multi-omics and phenotype association mining method based on the explainable autoencoder of the present application is shown in Figure 1 , the input is multi-omics data and prior knowledge masking matrix, and the output is feature-level explanation result and phenotypic classification decision result. The overall framework can be roughly divided into three parts: constructing a prior knowledge residual variational sparse autoencoder, constructing an interpreter, and establishing a residual network. The specific functions of each part are as follows:

[0045] (1) Prior knowledge residual variational sparse autoencoder. The input is multi-omics data and prior knowledge masking matrix, and the output is low-dimensional latent representation of multi-omics conforming to the probability distribution of multi-omics hidden space after training, which lays the foundation for subsequent multi-omics and phenotype association decision. In addition, the training of the autoencoder can be coupled with the interpreter to establish the basis for feature-level explanation.

[0046] (2) Interpreter. The interpreter is based on the method of additive feature attribution and is coupled with the autoencoder and the residual network. While realizing the phenotypic classification decision in the residual network, the influence degree of each feature on the phenotypic classification is given through the effect value calculation, and the feature-level explainability is obtained accordingly.

[0047] (3) Residual network. The present application constructs a deep residual neural network, the input is the multi-omics low-dimensional expression output by the autoencoder, and the output is the phenotype classification decision result.

[0048] The following takes breast cancer multi-omics and phenotype association mining as an example, combined with the process Figure 3 The specific embodiments of the present application are described in detail. It is particularly pointed out that the implementation scenario of the embodiment is breast cancer multi-omics and phenotype association mining, and the specific implementation scheme discussed is only used to illustrate the implementation mode of the present application, and does not limit the scope of the present application. The data used in this example is from the American Cancer Genome Atlas, which includes three different types of omics data (DNA methylation, mRNA and miRNA), a total of 600 samples, and each sample phenotype can be classified into one of five categories (Basal-like, HER2, Luminal A, Luminal B and Normal-like).

[0049] (1) Establish a priori knowledge residual variational sparse autoencoder. As shown in Figure 2 , first, based on the prior knowledge masking matrix and breast cancer multi-omics data (DNA methylation, mRNA and miRNA, feature dimensions are 20106, 20156 and 823 respectively), the connection structure of the nodes of each layer of the sparse encoder is masked, and the residual connection is established combined with formula (4). Second, based on formula (3), the output of the encoder is processed into a low-dimensional latent representation of the hidden space probability distribution. Third, a fully connected decoder is constructed, which is coupled with the encoder to build a complete priori knowledge residual variational sparse autoencoder. Finally, corresponding Figure 3 to the model training step, based on formula (5) and formula (6), the training is carried out to generate low-dimensional latent representation, and coupled with the interpreter.

[0050] The number of layers of the encoder in this step is 5, and the number of nodes is 41085, 16368, 4096, 1024 and 512 respectively. The number of layers and nodes of the decoder is consistent with the encoder.

[0051] (2) Based on formula (7) and formula (8), an interpreter is established, which is coupled with the residual network, low-dimensional latent representation and autoencoder for feature level explanation.

[0052] (3) Establish an 18-layer residual network to classify and predict breast cancer subtypes based on the 512-dimensional latent representation of (1).

[0053] After the above work is completed, for each sample, the breast cancer phenotype classification result (one of the five categories) of the sample can be obtained from step (3), and the importance of the input features can be sorted from step (2), so as to quantify the influence of each feature on the classification result, and realize the explainability of the result at the feature level. After the sample classification and feature importance sorting are completed, the breast cancer multi-omics and phenotype explainable association mining is realized in the present application. In the implementation example, the overall model has a classification accuracy of more than 83% for the samples, and the feature importance sorting result can also find potential biomarkers (such as AMY1A, COL11A2, and TMEM207) related to breast cancer at the feature level.

Claims

1.A method for multi-omics and phenotype association mining based on an interpretable autoencoder, characterized in that, The steps are as follows: Step (1) constructing a priori knowledge residual variational sparse encoder The multi-omics high-dimensional data is variational encoded in the form of combining priori knowledge and sparse coding, and a residual connection is added to improve the training difficulty of the entire autoencoder model; the input of the priori knowledge residual variational sparse encoder is multi-omics data and priori knowledge masking matrix, and the output is the encoding of multi-omics data; the specific steps are as follows: Firstly, the sparse encoder structure is built based on the full connection network encoder of the priori knowledge masking matrix; in the connection between the upper layer and the lower layer of the encoder, the traditional autoencoder adopts the full connection form for connection, as follows: y = f(W T x + b) (1) Where y is the output vector of the next layer, x is the input vector of the upper layer, W is the weight matrix of the neural network layer, b is the bias vector, and f is the activation function; the priori knowledge masking matrix M is introduced based on formula (1); M is a sparse matrix, and the elements in it take values of 0 or 1; in the signal pathway with biological priori knowledge, if the value is 0, the signal pathway has not been opened, and there is no universal contact between them; if the value is 1, it means that the existing knowledge pathway is connected; after introducing M, formula (1) is improved as follows: y = f((M * W) T x + b) (2) After formula (2) processing, the full connection encoder is improved to the priori knowledge sparse encoder; Secondly, the hidden space probability distribution mechanism is added to build the priori knowledge variational sparse encoder; the encoder first learns the mean vector μ and the variance vector σ, and then generates the low-dimensional latent representation z by combining the mean and variance with the Gaussian distribution, as follows: z = u + σ ∈ (3) ∈ is a random vector sampled from the Gaussian distribution N(0, I), and I is the unit matrix; Finally, the multiple residual connection mechanism is added to construct the priori knowledge residual variational sparse encoder; Suppose the encoder has n layers, and between the total input to the first layer output, the first layer output to the second layer output, and so on to the n-2 layer output to the n-1 layer output, each residual connection mechanism is added; for each residual connection, its principle is based on the following formula: representing the output of the residual module, x l is the input to the residual module, x l W s is the identity mapping, W s is a linear mapping matching the input and output dimensions, F is the residual mapping to be learned; Step (2) constructing the decoder and the priori knowledge residual variational sparse autoencoder; The decoder input is the encoding of the multi-omics data output by step (1), and the output is the reconstructed feature x out The decoder is coupled with the encoder established in step (1) to construct a priori knowledge residual variational sparse autoencoder, and after training, the low-dimensional latent expression of multi-omics is output; the specific process is as follows: Firstly, the full connection decoder is established, and the decoder is built based on the full connection network of formula (1), and the number of layers and the number of nodes of each layer are the same as the encoder, but the connection between the nodes does not use the masking matrix, and there is no residual connection mechanism between the layers; Secondly, the decoder is coupled with the encoder of step (1) to construct the priori knowledge residual variational sparse autoencoder; finally, the autoencoder is trained; the loss function of the training is as follows: Loss = ||x in - x out || 2 + KL[N(u, σ), N(0, I)] (5)​ x in KL is the divergence function, p(x) and q(x) are two different arbitrary polynomials, and N is the Gaussian function; the autoencoder is trained based on formulas (5) and (6), the training is performed together with the interpreter of step (3), and the interpretability of the autoencoder deep learning model is realized accordingly; in addition, the multi-omics low-dimensional latent expression As the input of step (4), the classification decision of multi-omics and phenotype is made; Step (3) constructing the interpreter based on additive feature attribution Let f 原 be the model to be explained, i.e. the autoencoder, g the explanation model, focusing on a local feature x 局 , g uses a simplified feature x 局 ' by a mapping function such that x 局 = h(x 局 '), thus ensuring g(x 局 ') ~ f 原 (h(x 局 ')); additive feature attribution explanation is a linear function of binary variables, as follows: Where M represents the number of simplified features, α i The effect value representing a characteristic is obtained by summing all characteristic effect values. 局 ′), so that g(x) 局 ′) Approaching f 原 (h(x 局 ′)) can be interpreted by explaining the model to understand f 原 Explain each feature in the text; In other words, the goal of an additive feature-attribution based interpreter is to construct a g that is sufficiently close to f 原 and explain the importance of features through their feature effect values a in equation (7), i.e., computing a is the final goal of the interpreter. α is calculated by minimizing the objective function ω, and the objective function is as follows: wherein represents a kernel function, τ(g) is a regularization term, and L is a square loss function; the meaning of equation (8) is that g approximates f 原 by simplifying a set of sample losses L weighted by the kernel function in the input space; after minimizing ω, g and f 原 achieve fitting, and calculate the characteristic effect value α through ρ x局 ′(α) to achieve the explanation of each feature; Step (4) constructing the residual network The input of the residual network is the multi-omics low-dimensional latent expression trained in step (2) The output is a phenotype classification decision result; the residual network used is a one-dimensional residual network composed of multiple residual blocks with different input and output dimensions; more specifically, each residual block is sequentially connected by two convolutional blocks and a residual connection, the source of the residual connection is the input of the first convolutional block, and the purpose is the output of the second convolutional block, the connection mode is based on formula (4), and the residual network is composed of a pooling layer, four residual network modules, a global average pooling layer, and a fully connected layer, which can realize phenotype classification decision after training.

Images