Image genetics correlation analysis method based on comparative learning and conditional generative network

Abstract

The invention discloses an image genetics correlation analysis method based on comparative learning and a conditional generative network, and the method comprises the steps: firstly, carrying out the nonlinear mapping of multi-modal image data to a potential space, and extracting the features of the image data; then, implementing comparative learning among the multi-modal image features to enhance feature embedding; in order to adaptively learn the importance of feature embedding, an attention mechanism is introduced to effectively fuse multi-modal image data; in addition, single nucleotide polymorphism (SNP) data is preprocessed into numeric data through a sample population mapping matrix, and then the SNP data is further encoded using a variational autoencoder to extract potential genetic features; in order to construct a non-linear correlation between nerve image data and genetic data, SNP data and demographic data are utilized, a dual-identification condition generative adversarial network is proposed to generate a multi-modal nerve image and a modal attention vector, and early diagnosis of brain diseases can be realized.

Description

Technical Field

[0001] The invention designs an imaging genetics association analysis method based on contrastive learning and conditional generative networks, and relates to the imaging genetics field of bioinformatics. Background Art

[0002] Alzheimer's disease (AD) is a progressive neurodegenerative disease with an insidious onset, clinically characterized by global dementia symptoms such as memory impairment, aphasia, apraxia, agnosia, and executive dysfunction. The underlying pathogenesis of Alzheimer's disease is still unclear, and there is currently no effective treatment. In addition, mild cognitive impairment (MCI) is an intermediate stage between normal aging and early AD, and 10-15% of MCI subjects deteriorate into AD each year. Given the incurability of this disease, it is crucial to take some measures in the early stages of AD and slow its progression through treatment.

[0003] As a brain disease, AD is mainly manifested in two aspects: one is the degeneration of brain structure and function at the macro level, and the other is the potential heritability at the micro level. At the macro level, modern imaging technologies, including magnetic resonance imaging (MRI) and positron emission tomography (PET), can use different technologies to capture the morphology and metabolic activity of the brain and generate multimodal neuroimaging data, thereby depicting the brain from different angles. These multimodal neuroimaging data provide different but complementary information and have been shown to provide a comprehensive understanding of stroke. At the micro level, Alzheimer's disease is also a genetically complex disease. Changes in genetic information, such as high-resolution markers of single nucleotide polymorphisms (SNPs) in genome-wide association studies (GWAS), can be further used to explore the pathogenic mechanisms of AD.

[0004] As an emerging topic, imaging genetics captures the potential relationship between neuroimaging and genetic data in an integrated way. This technology combined with genetic data aims to reveal the genetic basis of brain structure and function, and provide new insights into the phenotype and genetic characteristics of the brain. On this basis, the focus of this invention is to develop a neuroimaging genetic association analysis framework that simultaneously combines multimodal imaging data and genetic data to achieve early diagnosis of AD, which is very important and meaningful.

[0005] Existing imaging genetics research is mainly divided into: multivariate regression method, bivariate association analysis method, and machine learning and deep learning methods. Although the existing methods have achieved excellent prediction performance, they still have the following defects:

[0006] (1) The structural information contained in multimodal imaging data has not been fully utilized. This information is the key prior knowledge for screening features related to AD diseases. In addition, multimodal imaging data contain common, complementary and heterogeneous information, which poses a major challenge to multimodal imaging genetics association analysis.

[0007] (2) The association between multimodal imaging data and genetic data is complex, and this complex correlation is difficult to analyze using simple deep learning models.

[0008] (3) Considering that genetic data alone cannot directly diagnose the state of brain disease, and existing methods usually only use imaging data as endophenotypes to implement disease diagnosis, it is necessary to establish a multimodal imaging genetics association model to establish an effective link between imaging genetics and AD. Summary of the invention

[0009] In order to solve the technical problems existing in the above-mentioned background technology, the present invention designs an imaging genetics association analysis method based on contrastive learning and conditional generative network.

[0010] In order to achieve the above objectives, the present invention mainly includes three modules, namely, a multimodal image data fusion module, a multimodal image genetic data association analysis module, and a disease diagnosis module. The main steps include:

[0011] (1) In the multimodal image data fusion module, the multimodal neural image data is encoded, the latent features of different modal data are extracted, and the features are further refined using contrastive learning to enhance the embedding and learn the complementary information of different modalities. In order to effectively integrate multimodal image data, the attention mechanism is used to adaptively learn the importance of latent embedding and realize multimodal data fusion.

[0012] (2) In the multimodal imaging genetic data association analysis module, a dual-discriminant conditional generative adversarial network is proposed to learn the latent representation of SNP data, thereby assisting multimodal imaging analysis. On the one hand, SNP data is encoded by a variational autoencoder (VAE) to extract latent features. Next, combined with demographic data, the SNP latent features are input into the generator to generate MRI and PET image representations, thereby exploring the correlation between brain phenotypes and genotypes.

[0013] (3) In the disease diagnosis module, the generative network generates an integrated modality attention vector, which integrates SNP information into multimodal neuroimaging data, enabling early diagnosis of neuropsychiatric diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is the overall flow chart of the present invention;

[0015] Figure 2 It is a schematic diagram of multimodal image data fusion of the present invention;

[0016] Figure 3 A schematic diagram of association analysis based on a dual-identification generative adversarial network of the present invention;

[0017] Figure 4 A comparison chart of the disease state classification performance of the present invention in four classification tasks; DETAILED DESCRIPTION

[0018] The present invention proposes an imaging genetics association analysis method based on contrastive learning and conditional generative networks. The implementation process of the present invention will be described in detail below with reference to the accompanying drawings.

[0019] like Figure 1 As shown, the present invention designs an image genetics association analysis method based on contrastive learning and conditional generative network, which mainly includes four modules: multimodal image data fusion module, multimodal image genetic data association analysis module, and disease diagnosis module. Each link is described in detail below.

[0020] 1. Multimodal image data fusion module

[0021] The 618 subjects used in the method of the present invention are all from the Alzheimer's Disease Neuroimaging Initiative (ADNI) public database, including multimodal imaging data and genetic data. These subjects come from different disease states, including 193 normal people (NC), 213 early mild cognitive impairment (EMCI), 131 late mild cognitive impairment (LMCI) and 81 AD patients. In addition, the invention also obtains the demographic data of the sample, including gender, age, and years of education. Each subject has three modal data: T1-weighted MRI, PET imaging, and SNP data. For multimodal imaging data, the SPM software package is used for data preprocessing, and the average gray matter density and PET glucose utilization of 116 brain regions are obtained based on the automatic anatomical labeling (AAL) atlas. For genetic data, the Plink software package is used to preprocess all SNP data, and a total of 2703 SNPs are screened and coded as {0,1,2}, representing dominant homozygous genotypes, heterozygous genotypes, and recessive homozygous genotypes, respectively.

[0022] like Figure 2 As shown in Figure 2, there are complex nonlinear relationships between image data. In order to further obtain the potential features of multimodal image data, a deep autoencoder is used to achieve nonlinear mapping of data. Modality image data (Each mode has samples and features) are fed into the encoder to obtain the latent representation of the data The decoder network then uses the latent embedding as input to obtain the reconstructed multimodal neuroimaging data For parameter optimization, the reconstruction loss between the original image data and the reconstructed image data is: .

[0023] In order to further enhance the feature embedding extracted from MRI and PET imaging data, the normalized temperature scaled cross entropy (NT-Xent) loss in contrastive learning is used to achieve multimodal contrastive learning in a shared latent space. The goal of contrastive learning is to cluster the positive pairs between the MRI and PET imaging representations of the same sample, and to keep the mismatched negative pairs away from each other. The NT-Xent loss guides the projected representations of the two modalities to learn joint features. The loss value is calculated by combining the MRI imaging features and the corresponding PET imaging features of samples. When a positive pair is defined in a batch, the other The MRI imaging and PET imaging features of samples are defined as negative pairs. Therefore, the loss function of contrastive learning is calculated by using all positive sample pairs in a batch of data. and Given the definition:

[0024] ,

[0025] in represents the indicator function, if Then it returns 1. is a temperature parameter that controls how far apart the negative pairs are. represents the dot product between the normalized latent feature representations of MRI and PET and is defined as:

[0026] .

[0027] In the early diagnosis of brain diseases, since multimodal imaging data measures different and complementary information, their contributions to the diagnosis results are different. Therefore, the invention introduces an attention mechanism to adaptively learn the importance of latent embeddings and achieve data fusion. Specifically, the latent embeddings of MRI and PET imaging data are Implement nonlinear transformation and then use shared attention vector To get the relevant attention coefficient, it is defined as:

[0028] ,

[0029] ,

[0030] in represents the tanh activation function, represents the weight matrix, Represents the bias vector. Finally, the invention can fuse these embeddings to obtain the final fused embedding , expressed as:

[0031] ,

[0032] in Represents a fully connected layer, which is used to learn highly variable features of the latent representation.

[0033] 2. Multimodal imaging genetic data association analysis module

[0034] like Figure 3 As shown in the figure, the module consists of two parts: the SNP data representation part and the association analysis part based on the dual-discrimination conditional generative adversarial network.

[0035] The first is the SNP data representation part. Given the SNP data matrix ,in represents the sample size, Represents the dimension of SNP features. In order to enable the deep neural network to better process discrete SNP data, the categorical values ​​of SNPs are mapped to numerical form. Specifically, the SNP mapping matrix is ​​constructed according to the SNP distribution characteristics. , where the values ​​of the sth row and jth column of the mapping matrix are Defined as:

[0036] ,

[0037] in represents the Kronecker-delta function, if Returns 1 if true, otherwise returns 0. It is The gene The mapping matrix G constructed here is based on the sample population mapping, because the elements of G reflect the normalized frequencies of the three genotype mutants that appear in all samples, that is, Based on the mapping matrix, SNP vector By Embedded SNP vectors express:

[0038] .

[0039] For the mapped SNP data representation, a variational autoencoder (VAE) is used to further extract the latent feature representation so that it can better represent the genetic data. As a probabilistic generative model, VAE learns data mapping in an unsupervised manner. VAE mainly consists of two parts: the encoder network to learn the data distribution of the SNP embedding vector and the decoder network to achieve data reconstruction. Therefore, the encoder network is defined as:

[0040] ,

[0041] in and Represent the mean and variance of the data distribution respectively. Then, the potential distribution of SNP data can be sampled from the known data distribution :

[0042] , in represents a sample vector from a standard normal distribution, and represents the dot product operation. Afterwards, the decoder network reconstructs the latent representation of the SNP data :

[0043] .

[0044] In order to further explore the complex associations between multimodal imaging phenotypes and genotypes in the association analysis module, a new dual-discriminant conditional generative adversarial network (GAN) is proposed in combination with demographic information to generate attention vectors from genetic representations and further explore the intrinsic correlation between neuroimaging and genetic data. In order to enrich prior information and improve relevance, the invention innovatively incorporates demographic data, including gender, age, and years of education. The gender of the subject is processed using one-hot encoding, and other demographic values ​​are processed using data normalization. These normalized data are then spliced ​​into demographic vectors AD is a progressive degenerative disease that causes changes in the subject's brain structure, brain function, or disease state over time, while genetic data never changes. Therefore, it is very meaningful to combine genetic data with demographic data and explore the relationship between static SNP data and longitudinal demographic data.

[0045] However, multimodal imaging data is highly complex, and it is difficult to generate attention vectors based on multimodal imaging using only genetic data and demographic data. Therefore, a new dual-discrimination conditional GAN ​​is proposed to implicitly learn the distribution information of multimodal imaging data. Specifically, the generator is used to generate fake MRI imaging data. , fake PET imaging data , and integrated modality attention vector for AD diagnosis . The generator is defined as:

[0046] .

[0047] In addition, in order to further help the conditional GAN ​​model generate more realistic MRI and PET imaging data, the invention further proposes two adversarial discriminators and They learn the distribution of each modality data separately and estimate the probability of each true modality data input. The purpose is to distinguish fake MRI images from real MRI imaging data. The goal is to distinguish fake PET imaging from real PET imaging data. Based on the least squares conditional GAN, the training objective of the generator can be expressed as minimizing the following adversarial loss function:

[0048] .

[0049] The goal of the discriminator is to maximize the above formula. Through the adversarial process of the generator and the two discriminators, the difference between the probability distribution of the generated image and the two real image distributions will be reduced at the same time. In this process, genetic data is input into the generative network to generate MRI and PET imaging data, so that the network can capture multimodal neuroimaging-genetic associations.

[0050] 3. Disease diagnosis module

[0051] In order to achieve early diagnosis of brain diseases, the invention finally proposes a disease diagnosis module. This module mainly performs Hadamard product on the fused multimodal image representation and the genetically guided modal attention vector to achieve disease classification. The predicted label output by this module is defined as:

[0052] ,

[0053] in represents a classification network, Represents the Hadamard product operation. In this module, in order to achieve early diagnosis of brain diseases, multiple fully connected layers are used to classify disease states:

[0054] ,

[0055] in represents the softmax activation function of the fully connected layer, represents the weight matrix, Represents the bias vector in the classification network.

[0056] The present invention is trained in an end-to-end manner and optimized using a hybrid loss function. The reconstruction loss of the SNP representation module by VAE can be expressed as:

[0057] ,

[0058] Among them, ELBO represents the lower bound of evidence, which is mainly used to calculate the approximate posterior probability. Then, a dual discriminator conditional GAN ​​model based on least squares loss is also developed, and the loss of the generator network is defined as:

[0059] ,

[0060] in represents the least squares loss. In addition, the training loss of the corresponding discriminator network is defined as:

[0061] .

[0062] Finally, classification loss is used to achieve early diagnosis of the disease. In order to control the class imbalance problem, focal cross entropy loss is used as the classification loss:

[0063] ,

[0064] in , is a control factor used to focus on samples that are difficult to distinguish. In addition, represents the true label of the disease.

[0065] 4. The disease state classification experimental results of the present invention are as follows Figure 4 Shown

[0066] In this invention, some traditional machine learning methods and advanced deep learning methods are used as comparison methods. Traditional machine learning methods such as random forest (RF) and linear regression (LR) are used for disease diagnosis. For the above methods, multimodal neuroimaging data (such as ROI features of MRI and PET data) are connected with genetic data (such as SNP features) to implement sample classification. At the same time, two basic deep learning methods, multi-layer perceptron (MLP) and convolutional neural network (CNN), are also explored for disease classification. In addition, three advanced deep learning methods are compared with the method proposed in this article, including DCCA and DCCAE as deep typical association analysis methods, which realize nonlinear mapping of different modalities through neural networks to maximize the correlation coefficient of neuroimaging data and genetic data. For DCCA and DCCAE, the screened imaging data and genetic data are spliced ​​and input into the support vector machine to obtain the classification results. DGDL is a deep generation-discrimination learning model that nonlinearly projects genetic data into MRI imaging data to achieve association analysis and disease diagnosis.

[0067] Figure 4 The ROC curves of each method are shown for comparison. From these results, it can be seen that traditional machine learning methods have difficulty in processing high-dimensional multimodal data and have poor classification performance. Compared with LR, RF has better generalization ability. Among the above deep learning methods, basic MLP and CNN can effectively capture the nonlinear relationship of the data and process high-dimensional data, showing relatively high performance. In addition, from the comparison results of DCCA and DCCAE, the features selected by deep association analysis are more discriminative, which further proves that multi-view analysis has a stronger ability in detecting nonlinear associations between neuroimaging and genetic data. However, the performance of DCCAE is not as good as DGDL, which shows that the association analysis between imaging and genetic data contains key information for classification. It is worth noting that the classification performance of the invention is worse than that of other comparison methods in all classification tasks, further proving the role of the invention in the early diagnosis of brain diseases.

Claims

1. The present invention discloses an imaging genetics association analysis method based on contrastive learning and conditional generative networks, characterized in that: step: Step 1: Propose a multimodal data fusion algorithm: nonlinearly map the multimodal image data to the latent space to extract the features of the multimodal image data; introduce contrastive learning to enhance the latent feature embedding of the multimodal image data. In order to further adaptively learn the importance of feature embedding, an attention mechanism is proposed to effectively fuse the multimodal image data; Step 2: Propose a genetic data feature representation algorithm: Use the sample population mapping matrix to preprocess the single nucleotide polymorphism (SNP) data, and then introduce a variational autoencoder to encode the SNP data to extract genetic features; Step 3: Propose a multimodal imaging genetics association analysis module: Using SNP data and demographic data, a dual-discriminant conditional generative adversarial network is proposed to generate multimodal imaging data and modality attention vectors; Step 4: Propose a disease diagnosis module and use the modal attention vector to achieve early diagnosis of the disease and understand the pathogenic mechanism of the disease.

2. The imaging genetics association analysis method based on contrastive learning and conditional generative networks according to claim 1, characterized in that: In step 1: Step 2.1: Magnetic resonance imaging (MRI) and positron emission tomography (PET) data are first input into a deep encoder network to extract important features, and then a decoder network is used to reconstruct multimodal imaging data; Step 2.2: In order to further enhance the feature embedding of multimodal image data, the multimodal image data of the same sample is used as the positive sample pair, and the image data of different samples is used as the negative sample pair. Contrastive learning is performed on the shared latent space features so that the positive pairs are clustered together and the negative pairs are separated from each other. Step 2.3: Since multimodal image data contains complementary information, an attention mechanism is proposed to adaptively learn the importance of latent embedding, implement nonlinear mapping on the extracted features and use shared attention vectors to achieve multimodal data fusion.

3. The imaging genetics association analysis method based on contrastive learning and conditional generative networks according to claim 1, characterized in that: In step 2: Step 3.1: Since the SNP data consists of {0, 1, 2}, in order to better process the SNP data, a sample population mapping matrix is ​​constructed based on the SNP distribution characteristics, that is, the proportion of the original SNP data (0 / 1 / 2) in all subjects is calculated to replace the original value; Step 3.2: For the mapped SNP data, a variational autoencoder is proposed to extract the latent genetic feature representation, and a decoder network is used to reconstruct the original data.

4. The imaging genetics association analysis method based on contrastive learning and conditional generative networks according to claim 1, characterized in that: In step 3: Step 4.1: Propose to use clinical demographic data, including gender, age, and years of education, implement one-hot encoding on age data, and normalize other data, and splice the processed data into a demographic vector; Step 4.2: The encoded SNP data and demographic vectors are input into the generator network to generate fake MRI data and fake PET data; two discriminators are proposed to distinguish the generated MRI data from the real MRI data, and the generated PET data from the real PET data; Step 4.3: Through the adversarial process of the generator and the two discriminators, the difference between the probability distribution of the generated image and the two real image distributions will be reduced at the same time; in addition, the genetic feature representation is used to generate MRI image data, PET image data and modality attention vectors through the generative network, and the modality attention vector can capture the association between multimodal images and genetic data.

5. The imaging genetics association analysis method based on contrastive learning and conditional generative networks according to claim 1, characterized in that: In step 4: Step 5.1: In the disease diagnosis module, it is proposed to perform the Hadamard product on the fused multimodal image data and the generated modal attention vector to obtain the predicted label, and use the fully connected layer and softmax function to realize the classification of disease status, thereby achieving early diagnosis of the disease.

Images