Method and system for generating cross-modal conditions for genomics based on latent diffusion model

Abstract

The present invention discloses a method and system for generating cross-modal conditions for genomics based on a latent diffusion model, belonging to the field of bioinformatics. The method comprises the following steps: designing and constructing autoencoders for gene expression and gene mutation data, respectively, and performing model training; constructing a latent feature space based on the encoders for the gene mutation data and training an unconditionally guided latent diffusion model; constructing and training a cross-modal feature alignment network based on the two trained encoders; and jointly fine-tuning the unconditionally guided latent diffusion model and the cross-modal feature alignment network. The present invention solves the problem of missing sample modalities in multimodal learning. Through cross-modal feature alignment, the potential correlation between gene expression data and gene mutation data can be captured, enhancing the information flow between different modalities. The latent diffusion model effectively generates high-quality prediction data when mutation data is missing.

Description

Technical Field

[0001] The present invention belongs to the field of bioinformatics and relates to multimodal missing data generation, and in particular to a genomics cross-modal condition generation method and system based on a latent diffusion model (LDM). Background Art

[0002] The combined multi-omics analysis of genomics and transcriptomics is a core approach in modern biological and medical research, and is particularly significant in cancer research. Genomics primarily studies the genomic DNA of an organism, including gene sequences, mutations, structural variations, and genetic alterations. Genomic data provides the underlying genetic context of tumorigenesis, helping to understand how cancer drives abnormal cell proliferation and metastasis through genetic mutations. Gene mutations are key drivers of cancer development, potentially altering gene function, leading to uncontrolled cell growth or evasion of immune surveillance. Transcriptomics, on the other hand, focuses on gene expression levels, analyzing RNA transcription to reveal gene expression patterns in different cells or tissues. Transcriptomic data can reveal how genes respond to various physiological and pathological signals in the tumor microenvironment and how these changes drive tumor initiation and progression. The combined analysis of genomic and transcriptomic data can provide more comprehensive and insightful information on cancer biology. By analyzing the relationship between gene mutations and gene expression, researchers can uncover the molecular mechanisms of cancer development and gain a deeper understanding of how different gene mutations influence cancer cell proliferation, migration, and drug resistance by altering gene expression profiles. For example, certain driver mutations may lead to the overexpression or silencing of specific genes, thereby affecting the cell cycle, DNA repair mechanisms, or cell death pathways. This combined analysis helps identify tumor-associated driver genes, mutation hotspots, and possible therapeutic targets.

[0003] As a technology that integrates multiple data sources, multimodal learning can effectively integrate data from different omics levels, such as genomic data, transcriptomic data, and clinical data, thereby improving the predictive power and interpretability of the model. In the field of cancer, if multimodal learning is applied, it can achieve in-depth mining of complex biological data, not only capturing the impact of gene mutations on gene expression, but also learning potential biomarkers from different data sources, thereby guiding the implementation of precision medicine. This cross-domain data fusion method not only helps in the early detection of cancer, but also promotes the development of personalized treatment plans, improves treatment efficacy and reduces side effects. Therefore, the joint multi-omics analysis of genomics and transcriptomics, combined with multimodal learning, can provide strong support for cancer research and clinical applications.

[0004] Multi-omics joint analysis or multimodal learning based on omics data holds significant value in biomedical research, but one key challenge is the lack of data across sample modalities. This missing data can be caused by limitations in experimental techniques, high sequencing costs, uneven sample quality, different acquisition times for different modalities, and the difficulty in obtaining some omics data. For example, in cancer research, some patients may only have gene expression data but lack mutation or proteomic data. This results in incomplete data, hindering model training and inference capabilities. Current research focuses on generating single-modal data, such as synthesizing missing single-omics data using generative adversarial networks (GANs) or diffusion models. However, this approach fails to capture the corresponding relationships between samples in different modalities, making it difficult to maintain cross-modal consistency during joint analysis. Therefore, research on cross-modal data generation is of great significance. First, it can complement missing modality data, making multi-omics joint analysis more comprehensive and improving model robustness. Second, cross-modal data generation can be used for data augmentation, expanding training datasets, and improving model generalization. Furthermore, research based on cross-modal generation can explore potential relationships between different omics levels and promote the development of precision medicine. Therefore, exploring effective cross-modal data generation methods will provide new ideas for multi-omics analysis and multimodal learning. Summary of the Invention

[0005] The purpose of the present invention is to address the deficiencies of the existing technology and provide a genomics cross-modal condition generation method and system based on a potential diffusion model.

[0006] The object of the present invention is achieved through the following technical solutions: In a first aspect, a method for generating genomic cross-modal conditions based on a potential diffusion model is provided, comprising the following steps:

[0007] A variational autoencoder is constructed for gene expression data and gene mutation data, respectively, and trained to obtain a first encoder and a first decoder based on gene expression data, and a second encoder and a second decoder based on gene mutation data. A latent feature space is constructed based on the second encoder, and the parameters of the second encoder and second decoder are frozen to train an unconditionally guided latent diffusion model.

[0008] A cross-modal feature alignment network is constructed based on the attention mechanism. The parameters of the first and second encoders are frozen. The gene expression data, which is obtained based on the potential representation of the first encoder, is input into the cross-modal feature alignment network. The network is then aligned with the gene mutation data, which is obtained based on the potential representation of the second encoder, to train the cross-modal feature alignment network.

[0009] Jointly fine-tune the unconditionally guided latent diffusion model and the cross-modal feature alignment network, and use the output of the cross-modal feature alignment network as the guidance condition for the latent diffusion model;

[0010] When using gene expression data to generate corresponding gene mutation data, the gene expression data is passed through the trained first encoder and cross-modal feature alignment network in sequence, then input into the latent diffusion model for initial noise and iterative denoising, and decoded by the second decoder to generate gene mutation data.

[0011] Furthermore, the gene expression data is calculated using a TPM method and logarithmically transformed; the gene mutation data is assigned a value of 0 or 1, with 0 indicating no mutation and 1 indicating mutation. Preferably, before constructing the variational autoencoder for training, the gene expression data is normalized.

[0012] In some embodiments, the variational autoencoder includes an encoder and a decoder, the encoder includes multiple fully connected layers with gradually reduced dimensions and LeakyReLU activation function layers and dropout layers between layers, and the decoder includes multiple fully connected layers with gradually increased dimensions and LeakyReLU activation function layers between layers.

[0013] In some embodiments, when the variational autoencoder is trained, a reconstruction loss function and a KL divergence loss function are added as the loss function, and the reconstruction loss function adopts mean square error.

[0014] Furthermore, the potential diffusion model includes gradually adding noise to the potential representation of the gene mutation data obtained based on the second encoder until it is converted into pure noise, and gradually denoising the pure noise using a denoising neural network to obtain a clean potential representation.

[0015] Preferably, the denoising neural network adopts a U-Net structure, including a self-attention layer, a 1D convolution layer and layer normalization.

[0016] In some embodiments, when the latent diffusion model is trained, the loss function is the square of the L2 norm of the difference between the actual noise added at each step and the predicted noise of the denoising neural network.

[0017] In some embodiments, in the cross-modal feature alignment network constructed based on the attention mechanism, the input features are mapped to Q, K, and V respectively through linear layers, and multi-level features are captured through a multi-head attention mechanism, and are output after passing through the first normalization layer, the fully connected layer, and the second normalization layer in sequence.

[0018] In some embodiments, when the cross-modal feature alignment network is trained, the sum of an alignment loss function and a cosine similarity loss function is used as the loss function, and the alignment loss function uses a mean square error.

[0019] Preferably, during the joint fine-tuning, the guiding condition and the denoising neural network use a cross-attention mechanism to perform feature fusion.

[0020] In some embodiments, the loss function used in the joint fine-tuning is the square of the L2 norm of the difference between the true noise under the guidance condition and the predicted noise of the denoising neural network.

[0021] Preferably, the initialization noise is Gaussian noise, and the iterative denoising uses a trained denoising neural network to gradually denoise the Gaussian noise.

[0022] As a second aspect, the present invention also provides a genomics cross-modal condition generation system based on a potential diffusion model, comprising one or more processors for implementing the above-mentioned genomics cross-modal condition generation method based on a potential diffusion model.

[0023] As a third aspect, the present invention also provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, is used to implement the above-mentioned method for generating genomic cross-modal conditions based on a potential diffusion model.

[0024] Compared with the existing technology, the beneficial effects of the present invention are: the present invention can make full use of the pre-trained encoder model to extract multi-level gene expression and gene mutation representations in the cross-modal data generation task, and through cross-modal feature alignment, it can explore the potential correlation between gene expression data and gene mutation data, and promote information interaction between different modalities. Using the potential diffusion model, high-quality prediction data can be efficiently generated in the absence of mutation data. The genomics cross-modal condition generation system proposed in the present invention not only improves the efficiency of data utilization, but also makes up for the lack of experimental data, thereby optimizing the fusion quality of multimodal data. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic flow chart of a method for generating cross-modal conditions for genomics based on a potential diffusion model according to an embodiment of the present invention;

[0026] Figure 2 The structure diagrams of the autoencoder models for gene expression and gene mutation data provided by the embodiments of the present invention;

[0027] Figure 3 A structural diagram of the unconditionally guided potential diffusion model provided by an embodiment of the present invention;

[0028] Figure 4 A diagram of the cross-modal feature alignment network structure provided by an embodiment of the present invention;

[0029] Figure 5 A diagram showing the structure of a potential diffusion model using a cross-modal feature alignment network as a conditional guide provided by an embodiment of the present invention;

[0030] Figure 6A diagram of the denoising U-Net network structure provided by an embodiment of the present invention;

[0031] Figure 7 This is a schematic diagram of a hardware structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be described in detail below with reference to the accompanying drawings. The features of the following embodiments and implementations may be combined with each other unless they conflict. In the following embodiments, if there is a method not specified, it means that conventional methods in the art may be used.

[0033] The present invention proposes a genomics cross-modal condition generation method based on a potential diffusion model, such as Figure 1 As shown, the following steps are included:

[0034] (1) First, the gene expression data B and gene mutation data A (sample missing modal data) are preprocessed, and the preprocessed gene expression data and gene mutation data (Sample missing modal data, where m < n, representing the number of samples of gene mutation data and gene expression data, respectively, and Representing the data dimensions of gene expression data and gene mutation data respectively) to design and build an autoencoder and perform model training, respectively and .

[0035] Specifically, preprocessed gene expression data was calculated using TPM (transcripts per million) and logarithmically transformed to log2(TPM+1). Gene mutation data were assigned values ​​of 0 or 1, with 0 indicating no mutation and 1 indicating mutation. All of these gene expression data were normalized when input into model training.

[0036] Specifically, such as Figure 2 As shown, the autoencoder is designed and constructed for gene expression data B , based on the variational encoder architecture, where the encoder mainly consists of three fully connected layers, gradually reducing the dimension ( The dimensionality is reduced to 512, 256, and 128 dimensions in sequence), and LeakyReLU is used as the activation function to avoid gradient vanishing and alleviate the activation problem of sparse input, and dropout is used to prevent overfitting of high-dimensional small sample data; the decoder uses three fully connected layers to gradually increase the dimension (increased to 128, 256, 512 dimensions in sequence, and finally increased to dimension ), and LeakyReLU is used as the activation function. The encoder can well extract the features of gene expression data, and then reconstruct the gene expression data through the latent features to autoencode the Perform model training.

[0037] Specifically, such as Figure 2 As shown, an autoencoder is designed and constructed for gene mutation data A. , also based on the variational encoder architecture, where the encoder mainly consists of three fully connected layers, gradually reducing the dimension ( The dimensionality is reduced to 512, 256, and 128 dimensions in sequence), and LeakyReLU is used as the activation function to avoid gradient vanishing and alleviate the activation problem of sparse input, and dropout is used to prevent overfitting of high-dimensional small sample data; the decoder uses three fully connected layers to gradually increase the dimension (increased to 128, 256, 512 dimensions in sequence, and finally increased to dimension ), and LeakyReLU is used as the activation function. The encoder can well extract features of gene mutation data, and the decoder can well decode gene mutation data for data reconstruction, thereby training the autoencoder .

[0038] When training the model, the loss function used by the autoencoder training is The reconstruction loss function is used and KL divergence loss function , where the reconstruction loss The mean square error loss (MSE) is used to measure the original data x and the reconstructed data The degree of similarity between Indicates the number of samples, KL divergence loss Encourage the posterior distribution of the encoder output to be close to the prior distribution, avoid over-dispersion of the latent representation space and promote good latent structure learning, where represents the posterior distribution of the encoder output, z is the potential representation of the encoder output, represents the prior distribution, Represents the calculation function of KL divergence:

[0039] ;

[0040] ;

[0041] ;

[0042] (2) Encoder based on gene mutation data A Constructing latent feature space and using frozen parameters Train unconditionally guided latent diffusion models (LDMs).

[0043] Specifically, such as Figure 3 As shown, LDM is based on The operation is performed on the constructed potential feature space, including the forward diffusion process and the reverse denoising process, where is input The forward diffusion process transforms the clean data in the latent space into the gene mutation data before. Gradually add Gaussian noise and finally transform it into pure Gaussian noise ; The inverse denoising process is done by denoising neural network From noise The noise is gradually removed in the decoder to restore the clean data in the latent space. Generate data.

[0044] Specifically, the denoising U-Net network structure is as follows Figure 6 As shown in the figure, this denoising U-Net network mainly consists of a self-attention layer (Self-Attention), 1D convolution (Conv1d), and layer normalization, with LeakyReLU as the activation function. This network uses 1D convolution, which is suitable for processing genomic data or time series data and can preserve local dependencies. The self-attention layer allows the model to focus on global dependencies, which overcomes the limitations of the local receptive field of one-dimensional convolution and is suitable for processing cross-gene or cross-feature dependencies. Adding layer normalization (LayerNorm) after the Conv1d layer can stabilize model training. Feature compression is performed on the left side (downsampling path), and feature recovery is performed on the right side (upsampling path). Skip connections are made from the left to the right, allowing features from different layers to be directly transferred to preserve key information and prevent information loss.

[0045] Specifically, we train the unconditionally guided latent diffusion model LDM. During the training process, the encoder and decoder All model parameters are frozen, where the loss function It is the square of the L2 norm of the difference between the actual noise (the noise added at each step of the forward diffusion process) and the predicted noise:

[0046] ;

[0047] in, represents the real noise, Indicates that the t-th step denoising neural network is based on The predicted noise, represents the potential representation of the data at step t, For a given observation The noise of the neural network prediction is denoised in the next step t, using a time-conditioned U-Net model.

[0048] (3) Encoder trained based on gene expression data and the encoder trained with gene mutation data , design and build cross-modal feature alignment network , and freeze the two encoders to train the cross-modal feature alignment network .

[0049] Specifically, such as Figure 4 As shown, the cross-modal feature alignment network The model is implemented using a neural network based on an attention mechanism. This network first maps input features into Q (Query), K (Key), and V (Value) through three linear layers. A multi-head attention mechanism then captures multi-level features. Layer normalization is then performed, and the mapped features are output through a fully connected layer and layer normalization. A residual network structure is also used twice to preserve feature information during the two layer normalizations.

[0050] Specifically, during the model training process, first pass and Obtaining potential representations of gene expression and gene mutation data pairs and , then, Input cross-modal feature alignment network Get alignment features , then and Perform feature alignment. In model training, in order to better align cross-modal features, the mean square error (MSE) is used as the alignment loss function. , Indicates the number of samples and uses the cosine similarity loss function Strengthen the similarity between features and obtain the loss function used to train the cross-modal feature alignment network :

[0051] ;

[0052] ;

[0053] .

[0054] (4) Unconditionally guided latent diffusion model LDM and cross-modal feature alignment network Perform joint fine-tuning, where cross-modal features align the network The output of is used as the guiding condition.

[0055] Specifically, such as Figure 5 As shown, the unconditionally guided latent diffusion model LDM and cross-modal feature alignment network Perform joint fine-tuning (freeze encoder , encoder and decoder parameters), where the cross-modal feature alignment network The output of is used as the guiding condition, and the loss function used in the joint fine-tuning of the model is :

[0056] ;

[0057] in, represents the real noise, represents the noise predicted by the denoising neural network at step t under the guidance condition c, and the condition vector c is the gene expression data input encoder Get potential representation ,Then Then through the cross-modal feature alignment network Calculated.

[0058] Specifically, the feature fusion method of the guided condition c and the denoising U-Net network adopts the cross attention mechanism, where the conditional guided condition c is used as K, V, and the output feature of each layer of the U-Net network is used as Q.

[0059] (5) When using gene expression data to generate corresponding gene mutation data, firstly convert the preprocessed gene expression data into Using pre-trained gene expression encoders Output latent representation , and then input the jointly fine-tuned cross-modal feature alignment network Get the conditional vector c, and then run the conditional generation of the jointly fine-tuned latent diffusion model LDM to initialize noise and iterative denoising to obtain the potential representation of gene mutations , and finally use the pre-trained gene mutation decoder Decoding generates corresponding gene mutation data.

[0060] Specifically, the initialization noise is .

[0061] Corresponding to the aforementioned embodiment of the method for generating genomic cross-modal conditions based on a latent diffusion model, the present invention also provides an embodiment of a system for generating genomic cross-modal conditions based on a latent diffusion model.

[0062] See also Figure 7An embodiment of the present invention provides a genomics cross-modal condition generation system based on a potential diffusion model, including one or more processors for implementing a genomics cross-modal condition generation method based on a potential diffusion model in the above embodiment.

[0063] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0064] The embodiment of the genomic cross-modal condition generation system based on the potential diffusion model of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. This embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, if Figure 7 As shown, it is a hardware structure diagram of any device with data processing capability where the genomics cross-modal condition generation system based on the potential diffusion model of the present invention is located. Figure 7 In addition to the processor, memory, network interface and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware according to the actual functions of the device with data processing capabilities, which will not be described in detail.

[0065] The implementation process of the functions and effects of each unit in this embodiment is specifically described in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0066] For this embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only illustrative, in which the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Those of ordinary skill in the art can understand and implement it without expending creative work.

[0067] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for generating genomic cross-modal conditions based on a potential diffusion model in the above embodiment is implemented.

[0068] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0069] The above embodiments are intended only to illustrate the design concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. The scope of protection of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design concepts disclosed in the present invention are within the scope of protection of the present invention.

Claims

1. A genomics cross-modal condition generation method based on a latent diffusion model, characterized by: The following steps are involved: Constructing a variational autoencoder for gene expression data and gene mutation data respectively for training, thereby obtaining a first encoder and a first decoder based on gene expression data, and a second encoder and a second decoder based on gene mutation data; Constructing a latent feature space based on the second encoder, freezing the parameters of the second encoder and the second decoder, and training an unconditionally guided latent diffusion model; A cross-modal feature alignment network is constructed based on the attention mechanism. The parameters of the first and second encoders are frozen. The gene expression data, which is obtained based on the potential representation of the first encoder, is input into the cross-modal feature alignment network. The network is then aligned with the gene mutation data, which is obtained based on the potential representation of the second encoder, to train the cross-modal feature alignment network. Jointly fine-tune the unconditionally guided latent diffusion model and the cross-modal feature alignment network, and use the output of the cross-modal feature alignment network as the guidance condition for the latent diffusion model; When using gene expression data to generate corresponding gene mutation data, the gene expression data is passed through the trained first encoder and cross-modal feature alignment network in sequence, then input into the latent diffusion model for noise initialization and iterative denoising, and decoded by the second decoder to generate gene mutation data.

2. The method for generating cross-modal conditions for genomics based on a potential diffusion model according to claim 1, characterized in that: The gene expression data were calculated using the TPM method and logarithmically transformed; the gene mutation data were taken as 0 or 1, with 0 indicating no mutation and 1 indicating mutation.

3. The method for generating cross-modal conditions for genomics based on a latent diffusion model according to claim 1, characterized in that: The variational autoencoder includes an encoder and a decoder, the encoder includes multiple fully connected layers with gradually reduced dimensions and LeakyReLU activation function layers and dropout layers between layers, and the decoder includes multiple fully connected layers with gradually increased dimensions and LeakyReLU activation function layers between layers; when training the variational autoencoder, the sum of the reconstruction loss function and the KL divergence loss function is used as the loss function, and the reconstruction loss function uses the mean square error.

4. The method for generating cross-modal conditions for genomics based on a potential diffusion model according to claim 1, characterized in that: The potential diffusion model includes gradually adding noise to the potential representation of gene mutation data obtained based on the second encoder until it is converted into pure noise, and gradually denoising the pure noise using a denoising neural network to obtain a clean potential representation; the denoising neural network adopts a U-Net structure.

5. The method for generating cross-modal conditions for genomics based on a potential diffusion model according to claim 4, characterized in that: When the latent diffusion model is trained, the loss function is the square of the L2 norm of the difference between the actual noise added at each step and the predicted noise of the denoising neural network.

6. The method for generating cross-modal conditions for genomics based on a potential diffusion model according to claim 1, characterized in that: In the cross-modal feature alignment network constructed based on the attention mechanism, the input features are mapped to Q, K, and V respectively through the linear layer, and the multi-level features are captured through the multi-head attention mechanism, and are output after passing through the first normalization layer, the fully connected layer, and the second normalization layer in sequence; when training the cross-modal feature alignment network, the sum of the alignment loss function and the cosine similarity loss function is used as the loss function, and the alignment loss function adopts the mean square error.

7. The method for generating cross-modal conditions for genomics based on a potential diffusion model according to claim 4, characterized in that: During the joint fine-tuning, the guided condition and the denoising neural network use a cross-attention mechanism to perform feature fusion; the loss function used in the joint fine-tuning is the square of the L2 norm of the difference between the real noise under the guided condition and the predicted noise of the denoising neural network.

8. The method for generating cross-modal conditions for genomics based on a potential diffusion model according to claim 4, characterized in that: The noise initialization uses Gaussian noise, and the iterative denoising uses a trained denoising neural network to gradually denoise the Gaussian noise.

9. A genomics cross-modal condition generation system based on a latent diffusion model, characterized by: The method comprises one or more processors for implementing the genomics cross-modal condition generation method based on a potential diffusion model according to any one of claims 1 to 8.

10. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, it is used to implement the genomics cross-modal condition generation method based on the potential diffusion model according to any one of claims 1 to 8.

Images