A cross-modal translation method, a model building method, and a data augmentation method
By constructing a bidirectional cross-modal translation model and data augmentation methods, the problem of cross-modal translation of single-cell RNA sequencing and DNA methylation sequencing data was solved, achieving bidirectional cross-modal translation and preservation of biological features, thus expanding the application scenarios of multi-omics analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANKAI UNIV
- Filing Date
- 2026-02-12
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies make it difficult to simultaneously acquire single-cell RNA sequencing data and DNA methylation sequencing data in the same cell, which limits the application scope of single-cell multi-omics analysis methods. Furthermore, existing cross-modal translation methods suffer from unidirectionality, insufficient modeling of regulatory patterns, lack of assessment of biological relevance, and difficulty in training with small samples.
A variational autoencoder is used as the base of the cross-modal translation model. A bidirectional encoder module and a decoder module are constructed. Combined with the MoE submodule, self-attention mechanism and feature recalibration submodule, bidirectional cross-modal translation of scRNA-seq and scDNAm data is realized. Data augmentation methods are used to ensure the preservation of biological features and the accuracy of cross-modal conversion.
It enables bidirectional cross-modal translation of scRNA-seq and scDNAm data, expanding the application scenarios of multi-omics analysis. The generated prediction data retains the authenticity of biological structures and is suitable for downstream multi-omics joint analysis and mechanism mining.
Smart Images
Figure CN122135773A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the interdisciplinary field of bioinformatics and artificial intelligence, and in particular relates to a single-cell multi-omics cross-modal translation method, a single-cell multi-omics cross-modal translation model construction method, and a single-cell multi-omics data augmentation method. Background Technology
[0002] The development of single-cell sequencing technology has enabled the characterization of gene regulation and cellular heterogeneity at single-cell resolution. Among them, single-cell RNA sequencing (scRNA-seq) can reflect the state of cellular gene transcriptional activity, and single-cell DNA methylation sequencing (scDNAm) can reveal genomic epigenetic regulatory information.
[0003] However, in actual experiments, due to factors such as high experimental costs, limited initial sample sizes, and insufficient compatibility between different sequencing technologies, it is difficult to simultaneously obtain scRNA-seq and scDNAm data in the same cell. This results in a large amount of single-cell experimental data containing only single-modality information, limiting the application scope of multi-omics joint analysis methods. To address this issue, existing studies have proposed various cross-modal inference or data completion methods, such as the MAPLE method, which enables the prediction of RNA data from single-cell DNA methylation data. It constructs the correspondence between different modalities based on linear mapping, canonical correlation analysis, or simple regression models. The scCross method, by introducing a deep neural network to integrate chromatin accessibility, RNA, and methylation data, can predict missing modalities.
[0004] However, existing methods still have many shortcomings: First, they are one-way, only supporting one-way prediction from DNA methylation to gene expression, which limits the application of reverse inference scenarios; second, they lack modeling of regulatory patterns, mostly using globally shared mapping functions, making it difficult to capture the specific regulatory effect of DNA methylation on gene expression in different genomic regions or cell types; third, they lack assessment of biological relevance, with most methods focusing only on the accuracy of numerical fitting and ignoring the preservation of real biological structures, cell type-specific markers, and functional pathway signals in translation-generated data; and fourth, they are difficult to train with small samples, as the scarcity of paired multi-omics data makes deep learning models prone to overfitting and have poor generalization ability.
[0005] Therefore, there is an urgent need for a technical solution that can stably and accurately achieve cross-modal conversion between single-cell DNA methylation data and RNA data under limited paired sample conditions. Summary of the Invention
[0006] To address the aforementioned technical problems, this invention provides a cross-modal translation method, a model building method, and a data augmentation method, which are particularly suitable for cross-modal translation of scRNA-seq data and scDNAm data in single-cell multi-omics.
[0007] The technical solution adopted in this invention is as follows: Firstly, a single-cell multi-omics cross-modal translation method is provided, comprising the following steps:
[0008] Acquire scRNA-seq data and / or scDNAm data, and perform preprocessing;
[0009] The scRNA-seq data and / or the scDNAm data are subjected to feature mapping processing to obtain low-dimensional potential representations of RNA attributes and / or low-dimensional potential representations of DNAm attributes.
[0010] Perform cross-modal transformation processing on the low-dimensional potential representation of the RNA attribute and / or the low-dimensional potential representation of the DNA m attribute to preserve biological features, thereby obtaining low-dimensional features of the DNA m attribute and / or low-dimensional features of the RNA attribute.
[0011] The low-dimensional features of the DNAm attribute and / or the low-dimensional features of the RNA attribute are subjected to high-dimensional reconstruction processing to obtain scDNAm prediction data and / or scRNA-seq prediction data.
[0012] Further, the preprocessing of the scRNA-seq data includes the following steps:
[0013] The scRNA-seq data is organized into a first counting matrix, where each element of the first counting matrix represents the expression count of the gene in the corresponding cell.
[0014] The first counting matrix is subjected to library size normalization and Log1p logarithmic transformation;
[0015] Calculate the expression dispersion of each gene in the first counting matrix, and screen for highly variable genes according to a set ratio.
[0016] Further, the preprocessing of the scDNAm data includes the following steps:
[0017] The scDNAm data is organized into a second counting matrix, where each element of the second counting matrix represents the methylation rate of that region in the corresponding cell.
[0018] The genomic regions in the second counting matrix are divided using a sliding window, and the average methylation rate within each window is calculated.
[0019] Genome windows with low coverage are removed according to a set ratio, and the median imputation method is used to fill the missing parts.
[0020] The average methylation rate of the second counting matrix is normalized by maximum-min.
[0021] Secondly, a method for constructing a single-cell multi-omics cross-modal translation model is provided, including the following steps:
[0022] A variational autoencoder is used as the base for the cross-modal translation model;
[0023] A bidirectional encoder module is constructed to map scRNA-seq data and scDNAm data into dimension-aligned low-dimensional latent representations, respectively.
[0024] Construct a cross-modal translator module to achieve cross-modal conversion between two types of low-dimensional latent representations;
[0025] Construct an adversarial discriminator module to determine the authenticity of cross-modal latent representations;
[0026] A bidirectional decoder module is constructed, mirror-symmetric to the bidirectional encoder module, for reconstructing high-dimensional prediction data;
[0027] The cross-modal translation model is trained, and its parameters are optimized.
[0028] Furthermore, the bidirectional encoder module includes:
[0029] The RNA encoder submodule employs a multilayer perceptron architecture to map the high-dimensional gene expression features of the scRNA-seq data to a low-dimensional latent space.
[0030] The DNAm encoder submodule adopts a hierarchical architecture consisting of chromosome partitioning and global fusion, which is used to perform feature splicing after chromosome feature extraction on the scDNAm data.
[0031] The RNA encoder submodule and the DNA m encoder submodule have the same output dimension.
[0032] Furthermore, the cross-modal translator module includes:
[0033] The MoE submodule employs a parallel fusion architecture of expert subnetworks and gated networks to transform low-dimensional latent representations into weighted fusion features.
[0034] The self-attention mechanism submodule adopts a scaled dot product attention architecture to refine the weighted fusion features output by the MoE submodule.
[0035] The feature recalibration submodule adopts the SE feature recalibration architecture to adaptively amplify the feature dimension containing biological information and suppress the feature dimension with noise.
[0036] Furthermore, the MoE submodule includes:
[0037] Several parallel expert subnetworks are used to learn different subspaces of the data distribution;
[0038] A gating network is used to dynamically determine the suitability of each expert subnetwork to the current input based on the potential characteristics of the cell.
[0039] The weighted fusion unit is used to perform weighted fusion of the outputs of each expert subnetwork and the weights of the gated network.
[0040] Furthermore, training the cross-modal translation model includes the following steps;
[0041] Obtain the training dataset;
[0042] The parameters of the cross-modal translation model are initialized and the training parameter configuration is set;
[0043] The bidirectional encoder module and the bidirectional decoder module are pre-trained, and their parameters are saved.
[0044] Joint integration training is performed to enhance the cross-modal conversion capability of the cross-modal translation model and ensure the high fidelity and biological consistency of the translation data.
[0045] Furthermore, joint integration training includes the following steps:
[0046] The parameters of the generator are fixed, and the adversarial discriminator module is trained. The generator includes the bidirectional encoder module, the bidirectional decoder module, and the cross-modal translator module.
[0047] The generator is trained by fixing the parameters of the adversarial discriminator module;
[0048] Perform iterative training and convergence verification.
[0049] Thirdly, a method for enhancing single-cell multi-omics data is provided, including the following steps:
[0050] The original paired data were grouped according to cell type to ensure that cells in the same group have similar biological regulatory characteristics. The original paired data included the scRNA-seq data and the scDNAm data.
[0051] Within each cell type group, the scRNA-seq data map is kept unchanged, and the index of the scDNAm data map within that group is randomly permuted;
[0052] The synthetic paired sample is formed by combining the scRNA-seq data and the replaced scDNAm data;
[0053] The original pairing data and the synthetic pairing samples are mixed to obtain the training dataset.
[0054] The advantages and positive effects of this invention are as follows: by adopting the above technical solution, bidirectional cross-modal translation of scRNA-seq and scDNAm is realized, expanding the application scenarios of multi-omics analysis; the cross-modal conversion retains biological characteristics throughout, making the generated prediction data fit the biological structure of the real data, rather than simply numerical fitting; a complete processing link from raw data to prediction data is formed, and the high-fidelity prediction data output can be directly used for downstream multi-omics joint analysis and mechanism mining. Attached Figure Description
[0055] Figure 1 This is a flowchart illustrating a cross-modal translation method according to an embodiment of the present invention;
[0056] Figure 2 This is a flowchart illustrating a method for constructing a cross-modal translation model according to an embodiment of the present invention;
[0057] Figure 3 This is a schematic flowchart of a data augmentation method according to an embodiment of the present invention. Detailed Implementation
[0058] The present disclosure will now be described more fully with reference to the accompanying drawings, which illustrate exemplary embodiments of the present disclosure. The technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative effort are within the scope of protection of the present disclosure.
[0059] like Figure 1 As shown, this invention provides a single-cell multi-omics cross-modal translation method, comprising the following steps:
[0060] S11. Obtain scRNA-seq data and / or scDNAm data, and perform preprocessing;
[0061] Specifically, obtain scRNA-seq data (RNA expression) and scDNAm data (DNA methylation) from single-cell multi-omics data. Either scRNA-seq data or scDNAm data is sufficient, or both can be obtained. The scRNA-seq data and scDNAm data must meet the requirements of the same cell type, the same developmental stage, and the same experimental conditions. Then, preprocess the scRNA-seq data and scDNAm data separately to obtain a simplified "cell-hypervariate gene" matrix and a normalized "cell-genome region" matrix.
[0062] S12. Perform feature mapping processing on scRNA-seq data and / or scDNAm data to obtain low-dimensional potential representations of RNA attributes and / or low-dimensional potential representations of DNAm attributes.
[0063] Specifically, the preprocessed high-dimensional data obtained in step S11 undergoes feature mapping. When the acquired data is scRNA-seq data, the simplified matrix of "cell-hypervariate gene" is subjected to feature compression processing to retain core biological characteristics such as differences in gene expression strength and cell type specificity, while compressing the high-dimensional data into a low-dimensional feature vector, i.e., a low-dimensional potential representation of RNA attributes. When the acquired data is scDNAm data, the normalized matrix of "cell-genome region" is subjected to feature compression processing to retain core information such as methylation regulation characteristics of chromosome partitions and cell epigenetic status, while compressing the high-dimensional data into a low-dimensional feature vector, i.e., a low-dimensional potential representation of DNAm attributes.
[0064] Preferably, the low-dimensional potential representation of RNA attributes is a 128-dimensional low-dimensional feature vector; the low-dimensional potential representation of DNA m attributes is a 128-dimensional low-dimensional feature vector.
[0065] S13. Perform cross-modal transformation processing on the low-dimensional potential representation of RNA attributes and / or the low-dimensional potential representation of DNA m attributes to obtain low-dimensional features of DNA m attributes and / or low-dimensional features of RNA attributes.
[0066] Specifically, the low-dimensional feature vectors obtained in step S12 undergo cross-modal transformation. When the obtained low-dimensional feature vector is a potential low-dimensional representation of RNA attributes, low-dimensional DNA m-attribute features reflecting the epigenetic regulatory laws of the cell are generated through feature mapping and signal transformation. When the obtained low-dimensional feature vector is a potential low-dimensional representation of DNA m-attribute features, low-dimensional RNA attributes reflecting the gene transcriptional activity of the cell are generated through feature mapping and signal transformation. It is ensured that the dimensions of the two types of target low-dimensional features after transformation remain consistent with the dimensions of the original attribute low-dimensional features.
[0067] S14. Perform high-dimensional reconstruction processing on the low-dimensional features of DNAm attributes and / or low-dimensional features of RNA attributes to obtain scDNAm prediction data and / or scRNA-seq prediction data.
[0068] Specifically, the low-dimensional features obtained in step S13 undergo high-dimensional reconstruction. When the obtained low-dimensional features are DNAm attribute target low-dimensional features, a high-dimensional matrix of "cell-genome region" (i.e., scDNAm prediction data) is generated through feature expansion and signal reduction. When the obtained low-dimensional features are RNA attribute target low-dimensional features, a high-dimensional matrix of "cell-hypervariant gene" (i.e., scRNA-seq prediction data) is generated through feature expansion and signal reduction. scRNA-seq prediction data is synthetic data that simulates real scRNA-seq data. Its format and biological meaning are consistent with real scRNA-seq data, but its source is cross-modal conversion of scDNAm data, rather than direct experimental sequencing.
[0069] By employing the above method, the unidirectional limitations of existing methods are overcome, enabling bidirectional cross-modal translation of scRNA-seq and scDNAm, thus expanding the application scenarios of multi-omics analysis. The cross-modal conversion process retains biological characteristics throughout, ensuring that the generated predicted data closely matches the biological structure of the real data, rather than simply being a numerical fit. A complete processing chain from raw data to predicted data is formed, and the high-fidelity predicted data output can be directly used for downstream multi-omics joint analysis and mechanism mining.
[0070] To address the issues of varying sequencing depths between cells, unstable data variance, and interference from numerous low-information genes in scRNA-seq data, which necessitate targeted preprocessing to extract high-value gene features, this embodiment provides an implementation method.
[0071] In one embodiment, preprocessing scRNA-seq data includes the following steps:
[0072] The scRNA-seq data were organized into a first counting matrix, and the elements of the first counting matrix were the expression counts of the corresponding gene in the cells.
[0073] The first counting matrix is subjected to library size normalization and Log1p logarithmic transformation;
[0074] Calculate the expression dispersion of each gene in the first counting matrix, and screen for highly variable genes according to a set ratio.
[0075] Specifically, scRNA-seq data is organized into a "cell-gene" expression count matrix (first counting matrix), where rows represent individual cells, columns represent individual genes, and matrix elements represent the expression count of that gene in the corresponding cell. Library size normalization is applied to all gene expression counts for each cell in the matrix to eliminate expression level bias caused by differences in sequencing depth across different cells. The calculation equation is as follows: in This represents the normalized expression value of gene j in cell i. This represents the raw count of gene j in cell i within the unprocessed gene expression matrix. To calculate the total count for each cell i, a Log1p logarithmic transformation was performed on the normalized expression data. This aimed to stabilize the data variance, suppress interference from extremely overexpressed genes, and preserve the relative differences in gene expression. The calculation equation is as follows: in This represents the final expression value of gene j in cell i after logarithmic transformation. The expression dispersion of each gene is calculated, and hypervariable genes are screened according to a set ratio. Genes with stable expression levels and no biological differences are removed, retaining only the core gene characteristics that reflect cellular heterogeneity and functional differences, resulting in a simplified "cell-hypervariable gene" expression matrix.
[0076] The top 3000 hypervariable genes were selected.
[0077] Using the above methods, library size normalization eliminates expression level bias caused by different cell sequencing depths, making gene expression data comparable across cells; Log1p transformation effectively stabilizes data variance, suppresses interference from extremely high-expression genes, and preserves the relative differences in gene expression; screening for highly variable genes removes genes with no biological differences, focusing on core features that reflect cellular heterogeneity, laying a high-quality data foundation for subsequent feature mapping.
[0078] To address the issues of high sparsity, high noise, missing values, and genomic region coverage differences in scDNAm data, this embodiment provides an implementation method.
[0079] In one embodiment, preprocessing scDNAm data includes the following steps:
[0080] The scDNAm data were organized into a second counting matrix, and the elements of the second counting matrix were the methylation rates of the corresponding regions of the cells.
[0081] A sliding window was used to divide the genomic regions in the second counting matrix, and the average methylation rate within each window was calculated.
[0082] Genome windows with low coverage are removed according to a set ratio, and the median imputation method is used to fill the missing parts.
[0083] The average methylation rate of the second counting matrix is normalized by maximum-min.
[0084] Specifically, scDNAm data is organized into a "cell-genome region" methylation matrix (second counting matrix), where rows represent individual cells and columns represent specific regions on the genome (such as CpG islands or gene promoter regions). Matrix elements represent the methylation rate of that region in the corresponding cell. A sliding window is used to divide the genome into regions, and the average methylation rate within each window is calculated. This integrates discrete methylation site information into continuous region-level features, reducing data sparsity while preserving the spatial distribution characteristics of methylation. Genome windows with low coverage are removed according to a set ratio to avoid invalid features interfering with subsequent processing. Missing methylation values in the matrix are filled using median imputation. The median methylation rate of a single genome window is calculated across all valid cells, and this median replaces the missing values to ensure data integrity. The filled methylation rate data is scaled to a uniform range of [0,1] to eliminate inherent differences in methylation levels across different genome regions, making the data comparable across regions, resulting in a standardized "cell-genome region" matrix.
[0085] Preferably, the sliding window is set to 10kb; genomic windows with "coverage of less than 0.7% in all cells" are excluded.
[0086] Using the above method, sliding window aggregation integrates discrete methylation sites into continuous regional features, preserving the genomic spatial distribution characteristics of methylation and reducing data sparsity; removing low-coverage regions and imputing missing values at the median effectively improves data integrity and avoids interference from invalid features and null values in subsequent analysis; max-min normalization eliminates the inherent differences in methylation levels across different genomic regions, making the data comparable across regions and adapting to the needs of cross-modal feature alignment.
[0087] To address the shortcomings of existing cross-modal model architectures, such as their simple design, lack of modality specificity, inability to capture complex nonlinear regulatory relationships between single-cell multi-omics, and low fidelity and poor convergence of cross-modal generated data, this invention provides a method for constructing a single-cell multi-omics cross-modal translation model, comprising the following steps:
[0088] S21. A variational autoencoder is used as the base for the cross-modal translation model;
[0089] Specifically, the cross-modal translation model is based on a variational autoencoder (VAE) and is a modality-specific dual VAE architecture.
[0090] S22. Construct a bidirectional encoder module to map scRNA-seq data and scDNAm data into dimension-aligned low-dimensional potential representations, respectively.
[0091] S23. Construct a cross-modal translator module to realize cross-modal conversion of two types of low-dimensional latent representations;
[0092] S24. Construct an adversarial discriminator module to determine the authenticity of cross-modal latent representations;
[0093] Specifically, two modality-specific adversarial discriminator modules are constructed. Through adversarial training, the feature distribution generated by cross-modal translation is made closer to the real feature distribution, thereby improving the fidelity of the predicted data. The architecture of the adversarial discriminator module adopts a 3-layer fully connected network, with the dimension transformation link from 128 dimensions (input) to 64 dimensions (hidden layer 1) to 32 dimensions (hidden layer 2) to 1 dimension (output layer). The two discriminators (corresponding to the RNA modality and DNA m modality, respectively) have completely identical structures, with only their parameters being independent.
[0094] S25. Construct a bidirectional decoder module, which is mirror-symmetric to the bidirectional encoder module, for reconstructing high-dimensional prediction data;
[0095] Specifically, a bidirectional decoder module is constructed, which is structurally mirror-symmetric to the bidirectional encoder module, and is used to reconstruct high-dimensional prediction data. The bidirectional decoder module includes an RNA decoder and a DNAm decoder. The RNA decoder outputs the predicted gene expression profile, and the DNAm decoder outputs the predicted DNA methylation profile.
[0096] S26. Train the cross-modal translation model and optimize its parameters.
[0097] Using the above method, a variational autoencoder is used as the base, relying on its encoder-decoder architecture to ensure the model's ability to extract and reconstruct features from high-dimensional data, laying the foundation for cross-modal conversion. Multiple modules are functionally divided and collaboratively constructed to form a complete model chain of "encoding, conversion, discrimination, and decoding," adapting to the full-process requirements of cross-modal translation. Each module adopts a modality-specific design, conforming to the essential characteristics of scRNA-seq and scDNAm, improving the model's adaptability to the two types of data and the accuracy of cross-modal conversion.
[0098] To address the issue that existing encoders, with their globally shared architecture, fail to consider the essential differences between scRNA-seq and scDNAm, thus hindering effective feature extraction and cross-modal alignment, this embodiment provides an implementation method.
[0099] In one embodiment, the bidirectional encoder module includes:
[0100] The RNA encoder submodule employs a multilayer perceptron architecture to map high-dimensional gene expression features of scRNA-seq data to a low-dimensional latent space.
[0101] Specifically, the RNA encoder submodule adopts a 3-layer fully connected multilayer perceptron (MLP) architecture. The overall dimensionality transformation path is: 3000 dimensions (hypervariant gene input) to 256 dimensions (hidden layer 1) to 128 dimensions (output layer). The input layer receives the preprocessed "cell-hypervariant gene" simplified matrix and embeds 50% random mask units. During training, 50% of the elements in the input vector are randomly set to zero, forcing the model to rely on contextual features to recover complete information and avoid overfitting. Hidden layer 1 uses the LeakyReLU activation function with a negative slope of 0.01 to alleviate the gradient vanishing problem; a Dropout layer is then connected with a dropout rate of 0.1 to further suppress overfitting. The output layer has no activation function and directly outputs the low-dimensional latent representation of RNA attributes, ensuring the linear interactivity of features. The weights of all fully connected layers are initialized using Xavier, and the bias term is initialized to 0 to ensure gradient stability in the early stages of training.
[0102] The DNAm encoder submodule adopts a hierarchical architecture consisting of chromosome partitioning and global fusion, which is used to perform feature splicing after chromosome feature extraction from scDNAm data.
[0103] Specifically, the DNAm encoder submodule adopts a hierarchical architecture consisting of chromosome partitioning extraction and global fusion. The overall dimensionality transformation chain is as follows: N-dimensional (genome window input, where N is the number of genome windows per cell after preprocessing) to 32-dimensional / chromosome (local features) to concatenation to 128-dimensional (global output). The input layer receives the preprocessed "cell-genome region" normalized matrix (N-dimensional vector per sample), embedding 30% random mask units. During training, 30% of the elements in the input vector are randomly set to zero to improve the model's tolerance to high noise in methylated data. The chromosome partitioning extraction module splits the N-dimensional input vector into K chromosome sub-vectors according to chromosome number (K is the total number of chromosomes in the cell genome). Each sub-vector is mapped to 32-dimensional local features through a parameter-shared MLP sub-network (1 fully connected layer with ReLU activation), ensuring consistent feature extraction logic across different chromosomes while preserving partition specificity. The global fusion layer concatenates K 32-dimensional local chromosome features into a 32×K dimensional vector, which is then compressed to 128 dimensions through a fully connected layer, outputting a low-dimensional latent representation of DNA m attributes, strictly aligned with the output dimension of the RNA encoder. The weights of the shared MLP subnetwork and the global fusion layer are initialized using Xavier, with the bias term initialized to 0, consistent with the initialization strategy of the RNA encoder.
[0104] The RNA encoder submodule and the DNAm encoder submodule have the same output dimension.
[0105] Using the above methods, the RNA encoder adopts an MLP architecture, which is precisely adapted to the linear gene expression characteristics of scRNA-seq, achieving efficient compression and feature preservation of high-dimensional expression features; the hierarchical architecture of the DNAm encoder fits the genomic chromosome partitioning characteristics of methylation data, effectively extracting local features of each chromosome and fusing them into global features, preserving epigenetic spatial features; and the two encoders are forced to have the same output dimension, constructing a cross-modal unified latent space, enabling the two types of low-dimensional features to interact, providing a core prerequisite for subsequent cross-modal conversion.
[0106] To address the problem that existing cross-modal mapping functions are designed globally and cannot capture the specific regulation of different cell types and the long-distance dependence between features, and are easily affected by noise and lose core biological features, this embodiment provides an implementation method.
[0107] In one embodiment, the cross-modal translator module includes:
[0108] The MoE submodule employs a parallel fusion architecture of expert subnetworks and gated networks to transform low-dimensional latent representations into weighted fusion features.
[0109] The self-attention mechanism submodule adopts a scaled dot product attention architecture to refine the weighted fusion features output by the MoE submodule;
[0110] Specifically, the self-attention mechanism submodule adopts a scaled dot product attention architecture. The input is the 128-dimensional feature output from the MoE submodule, and the output is the 128-dimensional refined feature, which is then concatenated with the original input residual. The calculation steps are as follows: linear projection, through three independent fully connected layers (weight initialization uses the Xavier method), maps the 128-dimensional input feature into Query(Q), Key(K), and Value(V) matrices, each with a dimension of 128. The calculation equation is as follows: , , in , , These are the weights corresponding to Query (Q), Key (K), and Value (V). Attention weights are calculated using an equation. Calculation, where This is the output matrix of the self-attention mechanism. Let be the dimension of the key vector. Residual connections directly add the attention calculation results to the module input features, preserving the original feature information and improving the stability of model training. The calculation equation is: in This is the output of the self-attention mechanism submodule.
[0111] The feature recalibration submodule adopts the SE feature recalibration architecture to adaptively amplify the feature dimension containing biological information and suppress the feature dimension with noise.
[0112] Specifically, the feature recalibration submodule adopts the SE feature recalibration architecture. The input is the 128-dimensional output of the self-attention mechanism submodule, and the output is the 128-dimensional weighted and calibrated features. The calculation steps are: global average pooling (compression operation), where the 128-dimensional input features are subjected to global average pooling, compressing the features of each channel into a single scalar, resulting in a 128-dimensional channel descriptor. The equation is as follows: in The global average pooling value (channel descriptor) for the j-th channel represents the average activation of that channel across all spatial locations. The spatial dimension of the feature map. This is the element in the i-th row and j-th column of the self-attention output feature matrix. Channel attention (activation operation) learns channel weights through two fully connected layers, with a dimensionality transformation path from 128-dimensional to 128 / r-dimensional to 128-dimensional (r is the scaling ratio, defaulted to 16). The first layer uses ReLU activation, and the second layer uses Sigmoid activation, outputting a 128-dimensional channel weight vector. The output is recalibrated by multiplying the weight vector element-wise with the original input features, adaptively amplifying the channel weights of biologically effective features and suppressing noisy channels. The calculation equation is as follows: in The output features of the feature recalibration submodule are used to recalibrate the features. This is element-wise multiplication (Hadamard product). This is the channel weight vector.
[0113] Using the above methods, the MoE submodule can dynamically capture specific regulatory patterns of different cell types and different genomic regions, solving the problem of insufficient global mapping function modeling; the self-attention mechanism submodule can capture long-distance dependencies between multi-omics features, refine the fused features, and improve the integrity of feature representation; the SE feature recalibration submodule adaptively amplifies the feature dimension rich in biological information, suppresses noisy features, improves the model's noise resistance, and ensures the biological consistency of cross-modal transformation.
[0114] To address the problem that a single mapping function cannot adapt to the epigenetic-transcriptome regulatory patterns of different cell types, making it difficult to learn different subspaces of data distribution and resulting in insufficient modeling of specific regulatory relationships, this embodiment provides an implementation method.
[0115] In one embodiment, the MoE submodule includes:
[0116] Several parallel expert subnetworks are used to learn different subspaces of the data distribution;
[0117] Gated networks are used to dynamically determine the suitability of each expert subnetwork to the current input based on the potential characteristics of the cells.
[0118] The weighted fusion unit is used to weight and fuse the outputs of each expert subnetwork and the weights of the gating network.
[0119] Specifically, it consists of six parallel expert subnetworks, one gating network, and one weighted fusion unit. The input is a 128-dimensional low-dimensional latent representation, and the output is a 128-dimensional weighted fusion feature. Each of the six parallel expert subnetworks focuses on a different subspace of the data distribution. Each expert subnetwork's architecture employs a single fully connected layer (128-dimensional to 128-dimensional), a batch normalization (BN) layer, and a ReLU activation function. The momentum parameter of the BN layer is set to 0.99, and the epsilon is 1e-5 to improve training stability. The gating network architecture uses a two-layer MLP, with a dimensionality transformation path from 128-dimensional (input) to 64-dimensional (hidden layer) to 6-dimensional (output layer). The hidden layer uses ReLU activation, and the output layer uses Softmax activation, generating a 6-dimensional probability vector (the sum of all elements is 1), corresponding to the contribution weights of the six expert subnetworks. The weighted fusion unit fuses the outputs of the six experts using a weighted summation formula, calculated as follows: in The weighted sum of each expert subnetwork, Let be the contribution weight of the i-th expert subnetwork to the current output. For the i-th expert subnetwork, These are the input features of the MoE submodule.
[0120] Using the above method, multiple parallel expert sub-networks can learn different subspaces of the data distribution, with each sub-network focusing on the feature transformation of a specific cell type or regulatory pathway, thus improving the precision of modeling. The gating network can dynamically calculate expert weights based on the potential features of cells, achieving adaptive expert allocation for different cell inputs and adapting to context-dependent regulatory modes. The weighted fusion unit integrates the output features of each expert sub-network, making full use of the feature information of multiple subspaces, so that the transformed features are more consistent with the real cell regulatory features.
[0121] To address the issues of convergence difficulties and insufficient intra-modal feature learning that arise from the single-stage direct training often employed in existing models, this embodiment provides an implementation method.
[0122] In one embodiment, training a cross-modal translation model includes the following steps;
[0123] Obtain the training dataset;
[0124] Initialize the parameters of the cross-modal translation model and set the training parameter configuration;
[0125] The bidirectional encoder and bidirectional decoder modules are pre-trained, and their parameters are saved.
[0126] Specifically, the bidirectional encoder and decoder modules learn the internal feature patterns of their respective modalities, achieving a precise mapping from high-dimensional data to low-dimensional latent representations and then to high-dimensional reconstruction, laying a robust intra-modal foundation for cross-modal conversion. The scRNA-seq data from the training dataset is input into the RNA encoder, a 50% random mask is applied, and the output is a 128-dimensional low-dimensional latent representation of RNA attributes. This latent representation is directly input into the RNA decoder to reconstruct predicted scRNA-seq data. Backpropagation loss is applied to update the parameters of the RNA encoder and RNA decoder, while the translator and discriminator parameters are kept unchanged. This is achieved through equations... Calculate the loss function for this stage, where The loss function for the RNA pre-training phase. For the loss of the lower boundary of evidence, This is real scRNA-seq data. To predict scRNA-seq data, This represents the latent variables in scRNA-seq data, and MSE is the mean squared error loss, used for continuous value prediction.
[0127] The scDNAm data is input into the DNAm encoder, a 30% random mask is applied, and a 128-dimensional low-dimensional latent representation of DNAm attributes is output through a process of extracting attributes by chromosome partition and then global fusion. This latent representation is then input into the DNAm decoder to reconstruct and generate predicted scDNAm data. Backpropagation loss is applied to update the parameters of the DNAm encoder and DNAm decoder. The parameters of the remaining modules are fixed, and the equations are used to generate the predicted scDNAm data. Calculate the loss function for this stage, where For the DNAm pre-training stage, (This refers to the loss function.) For real scDNAm data, To predict scDNAm data, This represents the latent variables in scDNAm data. BCE is the binary cross-entropy loss, which is commonly used for binary or probabilistic data.
[0128] Joint integration training is conducted to enhance the cross-modal conversion capability of the cross-modal translation model and ensure the high fidelity and biological consistency of the translation data.
[0129] Using the above method, the encoder and decoder are trained independently in the pre-training stage, allowing the model to first master the feature extraction and reconstruction capabilities within a single modality and learn robust internal representations, laying the foundation for joint training. Joint integration training optimizes all modules in tandem, achieving parameter synergy among modules, effectively enhancing the model's cross-modal conversion capabilities and improving conversion accuracy. The two-stage training strategy avoids the convergence problem of single-stage training, ensuring the stability of model training, while ensuring that the generated translation data has both high fidelity and biological consistency.
[0130] To address the issues of pattern collapse and overpowered discriminator that can occur when the generator and discriminator are updated synchronously during adversarial training, this embodiment provides an implementation method.
[0131] In one embodiment, joint integration training includes the following steps:
[0132] The parameters of the generator are fixed, and the adversarial discriminator module is trained. The generator includes a bidirectional encoder module, a bidirectional decoder module, and a cross-modal translator module.
[0133] Specifically, the total loss of the adversarial discriminator module Through equations Calculation, where To counter discriminator models, For real data samples, Fake data samples generated by the generator. The labels are real data samples and follow a uniform distribution. , The labels of the fake data samples follow a uniform distribution. .
[0134] The generator is trained by fixing the parameters of the adversarial discriminator module;
[0135] Specifically, the total loss function The equation is as follows Calculation, where To reconstruct the loss, For translation losses, For KL divergence loss, To combat the losses.
[0136] Reconstruction loss Through equations Calculation, where The weights for RNA reconstruction loss. The weights for DNAm reconstruction loss, For RNA encoders, For the encoder of DNAm, For RNA decoders, For DNAm decoder, For RNA-to-RNA self-encoders, It is a DNAm-to-DNAm autoencoder.
[0137] Translation loss Through equations Calculation, where It is a cross-modal translator from DNA m to RNA. It is a cross-modal translator from RNA to DNA.
[0138] KL divergence loss Through equations Calculation, where This is an adjustable weighting coefficient used to control the importance of the KL divergence loss term in the total loss function.
[0139] Combating losses Through equations Calculation, where The weighting coefficients used to counteract the loss are used to adjust the proportion of the goal of the generator "deceiving" the discriminator in the overall optimization objective; This is the RNA modality synthesis data generated by the generator section. This is the DNAm modality synthesis data generated by the generator section.
[0140] Perform iterative training and convergence verification.
[0141] The above method employs an alternating training approach where one side is fixed while the other is updated, avoiding competition imbalance between the generator and discriminator, effectively preventing mode collapse, and ensuring the stability of adversarial training. Through adversarial training, the latent representation distribution generated across modalities approximates the real modal distribution, improving the fidelity of cross-modal transformations and making the generated data more consistent with the statistical characteristics of real data. Iterative training and convergence verification can monitor the model training status in real time, promptly judge the model convergence, avoid overfitting, and further improve the biological consistency of the translated data.
[0142] To address the scarcity of paired single-cell multi-omics data, existing random perturbation-based data augmentation methods are prone to losing biological features, leading to model overfitting and poor generalization and robustness in small sample scenarios. This invention provides a single-cell multi-omics data augmentation method, comprising the following steps:
[0143] S31. Group the original paired data according to cell type to ensure that cells in the same group have similar biological regulatory characteristics. The original paired data includes scRNA-seq data and scDNAm data.
[0144] S32. Within each cell type group, keep the scRNA-seq data map unchanged and randomly replace the index of the scDNAm data map within that group;
[0145] S33. Create a synthetic pairing sample of scRNA-seq data and the replaced scDNAm data;
[0146] S34. Mix the original pairing data and the synthetic pairing samples to obtain the training dataset.
[0147] Specifically, the original paired data is grouped according to cell type to ensure that cells within the same group have similar biological regulatory characteristics. Assuming that the original paired data contains K cell types, the paired datasets are first grouped according to cell type labels. Divide into K subsets For each subset Extract all its RNA maps DNA m map While maintaining the original order of the scRNA-seq data maps, a random permutation index was generated to rearrange the DNAm maps. This process generates a synthetic pairing sample of scRNA-seq data and the replaced scDNAm data. The original pairing data and the synthetic pairing sample are then mixed to obtain the training dataset.
[0148] Using the above method, data augmentation is performed based on cell type grouping. By relying on the similar regulatory characteristics of the same cell type, the biological consistency of the synthesized paired samples is ensured, and feature loss is avoided. Only the index of scDNAm data is replaced to match its high noise and high variability characteristics, while retaining the core gene expression characteristics of scRNA-seq to ensure data correlation. This effectively expands the training sample size, alleviates the problem of small sample training, and allows the model to learn common robust features within cell types, significantly improving the model's generalization ability and robustness in low-sample scenarios.
[0149] The embodiments of the present invention have been described in detail above, but the content described is only a preferred embodiment of the present invention and should not be considered as limiting the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the patent coverage of the present invention.
Claims
1. A cross-modal translation method, characterized in that, Includes the following steps: Acquire scRNA-seq data and / or scDNAm data, and perform preprocessing; The scRNA-seq data and / or the scDNAm data are subjected to feature mapping processing to obtain low-dimensional potential representations of RNA attributes and / or low-dimensional potential representations of DNAm attributes. Perform cross-modal transformation processing on the low-dimensional potential representation of the RNA attribute and / or the low-dimensional potential representation of the DNA m attribute to preserve biological features, thereby obtaining low-dimensional features of the DNA m attribute and / or low-dimensional features of the RNA attribute. The low-dimensional features of the DNAm attribute and / or the low-dimensional features of the RNA attribute are subjected to high-dimensional reconstruction processing to obtain scDNAm prediction data and / or scRNA-seq prediction data.
2. The cross-modal translation method according to claim 1, characterized in that, Preprocessing the scRNA-seq data includes the following steps: The scRNA-seq data is organized into a first counting matrix, where each element of the first counting matrix represents the expression count of the gene in the corresponding cell. The first counting matrix is subjected to library size normalization and Log1p logarithmic transformation; Calculate the expression dispersion of each gene in the first counting matrix, and screen for highly variable genes according to a set ratio.
3. The cross-modal translation method according to claim 1, characterized in that, Preprocessing the scDNAm data includes the following steps: The scDNAm data is organized into a second counting matrix, where each element of the second counting matrix represents the methylation rate of that region in the corresponding cell. The genomic regions in the second counting matrix are divided using a sliding window, and the average methylation rate within each window is calculated. Genome windows with low coverage are removed according to a set ratio, and the median imputation method is used to fill the missing parts. The average methylation rate of the second counting matrix is normalized by maximum-min.
4. A method for constructing a cross-modal translation model, characterized in that, Includes the following steps: A variational autoencoder is used as the base for the cross-modal translation model; A bidirectional encoder module is constructed to map scRNA-seq data and scDNAm data into dimension-aligned low-dimensional latent representations, respectively. Construct a cross-modal translator module to achieve cross-modal conversion between two types of low-dimensional latent representations; Construct an adversarial discriminator module to determine the authenticity of cross-modal latent representations; A bidirectional decoder module is constructed, mirror-symmetric to the bidirectional encoder module, for reconstructing high-dimensional prediction data; The cross-modal translation model is trained, and its parameters are optimized.
5. The method for constructing a cross-modal translation model according to claim 4, characterized in that, The bidirectional encoder module includes: The RNA encoder submodule employs a multilayer perceptron architecture to map the high-dimensional gene expression features of the scRNA-seq data to a low-dimensional latent space. The DNAm encoder submodule adopts a hierarchical architecture consisting of chromosome partitioning and global fusion, which is used to perform feature splicing after chromosome feature extraction on the scDNAm data. The RNA encoder submodule and the DNA m encoder submodule have the same output dimension.
6. The method for constructing a cross-modal translation model according to claim 4, characterized in that, The cross-modal translator module includes: The MoE submodule employs a parallel fusion architecture of expert subnetworks and gated networks to transform low-dimensional latent representations into weighted fusion features. The self-attention mechanism submodule adopts a scaled dot product attention architecture to refine the weighted fusion features output by the MoE submodule; The feature recalibration submodule adopts the SE feature recalibration architecture to adaptively amplify the feature dimension containing biological information and suppress the feature dimension with noise.
7. The method for constructing a cross-modal translation model according to claim 6, characterized in that, The MoE submodule includes: Several parallel expert subnetworks are used to learn different subspaces of the data distribution; A gating network is used to dynamically determine the suitability of each expert subnetwork to the current input based on the potential characteristics of the cell. The weighted fusion unit is used to perform weighted fusion of the outputs of each expert subnetwork and the weights of the gated network.
8. The method for constructing a cross-modal translation model according to claim 4, characterized in that, Training the cross-modal translation model includes the following steps; Obtain the training dataset; The parameters of the cross-modal translation model are initialized and the training parameter configuration is set; The bidirectional encoder module and the bidirectional decoder module are pre-trained, and their parameters are saved. Joint integration training is performed to enhance the cross-modal conversion capability of the cross-modal translation model and ensure the high fidelity and biological consistency of the translation data.
9. The method for constructing a cross-modal translation model according to claim 8, characterized in that, Joint integration training includes the following steps: The parameters of the generator are fixed, and the adversarial discriminator module is trained. The generator includes the bidirectional encoder module, the bidirectional decoder module, and the cross-modal translator module. The generator is trained by fixing the parameters of the adversarial discriminator module; Perform iterative training and convergence verification.
10. A data augmentation method for generating the training dataset in the method for constructing a cross-modal translation model as described in claim 8, characterized in that, Includes the following steps: The original paired data were grouped according to cell type to ensure that cells in the same group have similar biological regulatory characteristics. The original paired data included the scRNA-seq data and the scDNAm data. Within each cell type group, the scRNA-seq data map is kept unchanged, and the index of the scDNAm data map within that group is randomly permuted; The synthetic paired sample is formed by combining the scRNA-seq data and the replaced scDNAm data; The original pairing data and the synthetic pairing samples are mixed to obtain the training dataset.