A rice-oriented enhancer strength prediction and generative design method

Abstract

The present application relates to the technical field of enhancer strength prediction and generation, and in particular to a rice-oriented enhancer strength prediction and generation design method, which comprises the following technical solutions: a unified bioinformatics pipeline is used to reprocess and conditionally normalize multiple sets of rice STARR-seq data, and a benchmark data set is constructed; a pre-trained plant DNA language model is used to build a rice enhancer activity predictor, and the activity of the enhancer sequence is realized; a rice enhancer diffusion generator based on the U-Net architecture is introduced, which generates diversified candidate sequences from random noise, and forms a linkage screening mechanism with the rice enhancer activity predictor score; the influence of data heterogeneity on model training is eliminated, the model generalization ability and biological rationality under small sample conditions are improved, efficient exploration and novelty generation of sequence space are realized, de novo design and functional verification of synthetic enhancers are completed for the first time in crops, and the functional diversity and exploration efficiency of the generated sequences are enhanced.

Description

Technical Field

[0001] This invention relates to the field of enhancer strength prediction and generation technology, and in particular to a method for enhancing enhancer strength prediction and generative design for rice. Background Technology

[0002] Enhancers are a key class of cis-regulatory elements in eukaryotic genomes, capable of precisely activating the expression of target genes spatiotemporally by binding to transcription factors and their co-activators. The activity of enhancers determines the intensity and pattern of gene expression, serving as a core regulatory hub connecting genome sequence and phenotypic output. In the context of crop breeding entering the "Agriculture 4.0" era and traditional breeding strategies relying on natural variation gradually approaching the bottlenecks in yield and stress resistance improvement, the rational design and de novo synthesis of regulatory elements such as enhancers have become a key technological pillar in precision breeding and plant synthetic biology.

[0003] In recent years, high-throughput parallel reporter assays, exemplified by STARR-seq (Self-Transcribing Active Regulatory Region Sequencing), have enabled researchers to quantitatively determine the enhancer activity of hundreds of thousands of DNA fragments at the whole-genome scale, providing a high-quality data foundation for constructing sequence-activity mappings. In model organisms such as humans and fruit flies, deep learning-based discriminative models, such as DeepSTARR, Enformer, and BPNet, have successfully achieved high-precision predictions from DNA sequences to enhancer activities, significantly improving the ability to interpret cis-regulatory "grammar." However, these models are essentially still discriminative architectures, primarily used for scoring the activity of existing sequences, lacking the ability to actively explore the sequence space and generate entirely new functional sequences.

[0004] In the crop field, especially in cereal crops such as rice, the rational design of enhancers still faces the following technical bottlenecks:

[0005] The data foundation is weak and highly heterogeneous: Although some STARR-seq data have been published, there are significant differences between different studies in terms of library construction, sequencing depth, and activity normalization. The lack of a unified reprocessing process and a standardized activity benchmark set limits the construction of high-quality training data and cross-study comparisons.

[0006] Discriminative models dominate, lacking generative design capabilities: Current deep learning models in the plant field are mostly focused on enhancer classification or activity regression tasks, and have not yet formed a generative framework embedded in the genomic context and oriented towards de novo design. Traditional optimization methods, such as genetic algorithms, are prone to getting trapped in local optima and are difficult to efficiently explore the vast sequence space.

[0007] Lack of an integrated "prediction-generation" framework for crop systems: In human and animal studies, generative models such as diffusion models and language models have been initially applied to the de novo design of regulatory elements, but in crop systems, there is still a lack of robust and verifiable integrated frameworks, which limits the practical application of synthetic enhancers in molecular design breeding.

[0008] In summary, existing technologies have significant shortcomings in terms of data foundation, model capabilities, and design paradigms. There is an urgent need to build a computational framework for rice enhancers that combines high-precision prediction and generative design capabilities in order to overcome the current technical bottlenecks in the engineering design of cis-regulatory elements.

[0009] In view of this, we propose a rice-oriented enhancer strength prediction and generative design method to solve the existing problems. Summary of the Invention

[0010] The purpose of this invention is to provide a method for predicting and generatively designing enhancer strength in rice, in order to solve the problems mentioned in the background art.

[0011] To achieve the above objectives, the present invention provides the following technical solution: a method for predicting and generatively designing enhancer intensity in rice, comprising: first, reprocessing and conditionally normalizing multiple sets of rice STARR-seq data through a unified bioinformatics pipeline to construct a benchmark dataset; second, building a rice enhancer activity predictor using a pre-trained plant DNA language model to achieve regression of enhancer sequence activity; and third, introducing a rice enhancer diffusion generator based on the U-Net architecture to generate diverse candidate sequences from random noise, and forming a screening mechanism linked with the rice enhancer activity predictor score.

[0012] The steps for constructing and training the rice enhancer activity predictor include sequence segmentation and backbone representation, lightweight transfer fine-tuning strategy, learnable layer mixing and multi-scale pooling regression head, reverse complementary equivariant inference, and composite loss training; the steps for the denoising diffusion of the rice enhancer diffusion generator to generate enhancers from scratch include sequence continuous representation, forward diffusion definition, denoising network training, and reverse denoising sampling.

[0013] Among them, the rice enhancer activity predictor guides heteroscedastic uncertainty diffusion sampling, introduces uncertainty noise in the back diffusion, and at each back time step, the candidate sequence characterization is first obtained from the current state, input into the rice enhancer activity predictor to obtain the variance of the candidate, and the variance is used to adjust the scale of random noise injection in this step.

[0014] Furthermore, in the sequence segmentation and backbone representation steps, the 512 bp sequence is segmented using 6-mer and input into a pre-trained plant DNA language model to extract contextual representations.

[0015] Furthermore, in the lightweight migration fine-tuning strategy step, low-rank adaptation is introduced into the backbone attention module and fine-tuned in conjunction with partial layer unfreezing.

[0016] Furthermore, in the steps of learnable layer mixing and multi-scale pooling regression head, the hidden states of several layers at the top of the backbone are fused using learnable weighted fusion to obtain fused features, which are then entered into the multi-scale pooling module to compute and concatenate multiple pooling representations in parallel to obtain the global vector representation of the enhancer.

[0017] Furthermore, in the step of reverse complementary equivariant inference, the forward chain and the reverse complementary chain of the input sequence are encoded in parallel, and the two outputs are fused before and after the regression head to ensure reverse complementary equivariance.

[0018] Furthermore, in the training step of composite loss, heteroscedasticity loss based on Gaussian negative log-likelihood is adopted, and scale matching loss, consistency regularization, and L2 regularization can be superimposed.

[0019] Furthermore, in the step of sequence continuous representation, the 512 bp sequence is one-hot encoded and linearly scaled to a continuous space of [−1,1] as the diffusion training input.

[0020] Furthermore, in the forward diffusion definition step, a forward diffusion Markov chain with discrete time steps is defined, and a preset noise schedule is used to gradually perturb the real sequence to approximately Gaussian noise.

[0021] Furthermore, in the training step of the denoising network, a one-dimensional ResNet-style U-Net is constructed as the denoising network to predict noise and velocity parameters at each time step. During training, a weighted strategy is adopted to improve numerical stability and convergence speed.

[0022] Furthermore, in the reverse denoising sampling step, reverse denoising is iteratively performed starting from standard Gaussian noise using exponential moving average weights.

[0023] Compared with the prior art, the beneficial effects of the present invention are:

[0024] 1. This invention addresses the significant heterogeneity in existing rice STARR-seq data regarding library construction, sequencing depth, and activity normalization by establishing a unified bioinformatics processing workflow for the first time. By reprocessing two sets of publicly available data using the same alignment parameters, statistical models, and significance thresholds, and retaining consistent enhancer fragments from both biological replicates, 13,414 non-redundant 512bp high-confidence enhancer sequences were ultimately obtained. This dataset not only eliminates batch effects but also provides a large-scale, high-signal-to-noise-ratio standardized training foundation for quantitative modeling of rice enhancers, effectively solving the problem of a lack of a unified enhancer strength benchmark set in previous techniques. It constructs a high-quality, standardized rice enhancer activity benchmark dataset, effectively eliminating the impact of data heterogeneity on model training.

[0025] 2. The rice enhancer activity prediction model constructed in this invention employs a 6-mer segmentation strategy, a pre-trained Plant-DNABERT backbone network, and a customized regression head. In five-fold cross-validation grouped by chromosome, it achieved a PCC of 0.72 ± 0.01 and an R-value of [missing value]. 2 It exhibits excellent performance with a mean squared error (MSE) of 0.50 and an MSE of 0.15. Compared to BPE-encoded and single-base-encoded models, the 6-mer-encoded model shows improvements in PCC of approximately 4.3% and 2.9%, respectively. In a systematic comparison with 15 model variants, including Plant-DNA Mamba, Plant-DNA GPT, and Plant-Nucleotide Transformer, the rice enhancer activity prediction model demonstrates superior performance in PCC, SCC, and RSE. 2 It ranked first in all four metrics (MSE), especially in goodness of fit, leading the second-best model by more than 0.04, while reducing MSE by 0.01-0.05. This technical performance demonstrates that the rice enhancer activity prediction model can learn enhancer regulatory syntax from sequences with higher accuracy and generalization ability, providing a reliable activity scorer for subsequent generative design. It achieves high-precision quantitative prediction of enhancer activity based solely on DNA sequences, significantly outperforming existing deep learning models for plant genomes.

[0026] 3. To address the bottleneck of limited labeled data in the crop domain, this invention employs a hybrid differential fine-tuning strategy combining LoRA (low-rank adaptation) and high-level thawing. Ablation experiments show that the model performance is optimal (PCC ≈ 0.72) only when LoRA and backbone local thawing are enabled simultaneously, while the PCC is only approximately 0.69 and 0.68 when LoRA is used alone or only high-level thawing is used. Furthermore, the composite loss function designed in this invention (including Gaussian negative log-likelihood loss, scale-matching loss, inverse complementary consistency loss, and log-variance regularization) improves the PCC by approximately 0.04 compared to the traditional Huber loss on the validation set, effectively solving the problems of uncertainty modeling and dynamic range compression in quantitative regression. The inverse complementary isovariance design further enhances the model's utilization of the biological prior of DNA double-strand symmetry, improving the robustness of predictions. Through innovative fine-tuning strategies and composite loss functions, the model's generalization ability and biological rationality under small sample conditions are significantly improved.

[0027] 4. This invention abandons traditional genetic algorithms or local perturbation search strategies, and for the first time applies the Denoising Diffusion Probability Model (DDPM) to the de novo design of rice enhancers. The rice enhancer diffusion generator adopts a 1D U-Net architecture, combined with v-prediction parameterization and a min-SNR weighting strategy. After 2400 training epochs, the generated enhancer sequences show R0 values ​​similar to the natural sequences in the 4-mer, 5-mer, and 6-mer frequency distributions. 2 The consistency values ​​reached 0.96, 0.93, and 0.85 respectively, approaching the consistency level of the natural sequences (0.98, 0.97, and 0.93). Simultaneously, the minimum Jaccard distance and minimum Levenshtein distance between the generated sequences and the natural sequences remained stable and did not return to zero after convergence, indicating that the generated sequences possess significant novelty at the global sequence level while preserving local motif features. The GC content distribution also showed high consistency with the natural enhancers, avoiding statistical drift during the generation process. These technical effects demonstrate that the rice enhancer diffusion generator successfully learned the high-dimensional probability distribution manifold of the natural enhancers, enabling the generation of candidate sequences that combine biological plausibility and sequence innovation. A rice enhancer de novo generator based on a denoised diffusion probability model was constructed, achieving efficient exploration and novelty generation in the sequence space.

[0028] 5. This invention couples a rice enhancer activity predictor and a rice enhancer diffusion generator into a unified RED-T framework, forming a closed loop of "generation-prediction-screening". The rice enhancer diffusion generator generates a large number of candidate sequences from random noise, and the rice enhancer activity predictor performs high-throughput activity prediction on them, effectively expanding the sequence search space to regions not yet explored by natural evolution, while simultaneously screening high-confidence candidates through activity thresholds. Experimental evidence shows that the synthetic enhancers generated and selected by this framework have significant biological activity at the protoplast level. This technology fills the technological gap in crop systems from functional genomic data to de novo enhancer design, realizing a paradigm shift from "reading" regulatory codes to "writing" programmable regulatory elements.

[0029] 6. In the reverse denoising process of the rice enhancer diffusion generator, this invention innovatively introduces the posterior variance predicted by the rice enhancer activity predictor as a regulating factor: increasing noise injection in high uncertainty regions enhances the exploration capability of the sequence space; reducing noise in low uncertainty regions ensures stable convergence to the functional sequence. This mechanism effectively balances the diversity and activity potential of the generated sequences, avoiding the problem of traditional generation models being too conservative and unable to escape local optima, and providing a new technical means for the efficient design of synthesized enhancers. Introducing noise adjustment guided by prediction uncertainty during diffusion sampling enhances the functional diversity and exploration efficiency of the generated sequences. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the first part of the network structure of the rice enhancer activity prediction model of the present invention;

[0031] Figure 2 This is a schematic diagram of the second part of the network structure of the rice enhancer activity prediction model of the present invention;

[0032] Figure 3 This is a schematic diagram of the first part of the network structure of the rice enhancer diffusion generation model of the present invention;

[0033] Figure 4 This is a schematic diagram of the second part of the network structure of the rice enhancer diffusion generation model of the present invention;

[0034] Figure 5 This is a schematic diagram of the third part of the network structure of the rice enhancer diffusion generation model of the present invention;

[0035] Figure 6 This is a schematic diagram providing an overview of the RED-T computing framework of the present invention;

[0036] Figure 7 The first figure is for evaluating the convergence of the training process of the rice enhancer diffusion generation model of the present invention and the global sequence features of the generated sequence.

[0037] Figure 8 The second figure shows the convergence of the training process of the rice enhancer diffusion generation model of the present invention and the global sequence feature evaluation of the generated sequence.

[0038] Figure 9 The third figure shows the convergence of the training process of the rice enhancer diffusion generation model of the present invention and the global sequence feature evaluation of the generated sequence.

[0039] Figure 10 Figure 4 shows the convergence of the training process of the rice enhancer diffusion generation model of the present invention and the global sequence feature evaluation of the generated sequence.

[0040] Figure 11 Figure 5 shows the convergence of the training process of the rice enhancer diffusion generation model of the present invention and the global sequence feature evaluation of the generated sequence;

[0041] Figure 12 Figure 1 shows the prediction performance and ablation analysis of the rice enhancer activity prediction model of the present invention under different coding strategies, backbone networks and training configurations.

[0042] Figure 13 Figure 2 shows the prediction performance and ablation analysis of the rice enhancer activity prediction model of the present invention under different coding strategies, backbone networks and training configurations.

[0043] Figure 14 Figure 3 shows the prediction performance and ablation analysis of the rice enhancer activity prediction model of the present invention under different coding strategies, backbone networks and training configurations. Detailed Implementation

[0044] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0045] Example 1

[0046] In eukaryotes, cis-regulatory elements (CREs), such as promoters and enhancers, determine the spatiotemporal precision of gene expression and serve as crucial interfaces connecting genome sequences and phenotypic outputs. They largely constitute the genome's "operating system," maintaining individual development and environmental responses. As crop breeding enters the "Agriculture 4.0" stage, traditional strategies relying solely on natural variation are gradually approaching the theoretical limits of yield enhancement and stress resistance. To achieve breakthroughs with limited genetic resources, breeding thinking urgently needs to evolve from passively "reading" genomic information to actively "writing" programmable regulatory networks. Mastering the grammar of cis-regulation and designing enhancers with preset strength and tissue specificity at the DNA sequence level (de novo) has become a key fulcrum for precision breeding and plant synthetic biology. For staple crops such as rice (Oryza sativa), transforming enhancers from "passive annotation objects" into "engineerable design modules" constitutes a key breakthrough in improving the efficiency and accuracy of molecular design breeding, and is also one of the important theoretical and practical foundations of this application.

[0047] Large-scale parallel reporter experiments (MPRAs), exemplified by Self-Transcribing Active Regulatory Region sequencing (STARR-seq), combined with multidimensional epigenomics assays, have provided an unprecedentedly rich empirical data foundation for decoding the "grammar" of cis-regulation. These high-throughput technologies not only depict open chromatin states at the genome scale but also enable quantitative assessment of enhancer activity in a unified experimental system for tens of thousands of DNA fragments, thus establishing a direct "sequence-activity" mapping. In model species such as humans and fruit flies, deep learning architectures like DeepSTARR, Enformer, and BPNet have demonstrated their ability to capture complex motif combinations and long-range dependencies from such data, achieving high-precision sequence-to-activity prediction. However, these models, in essence, still primarily act as discriminative "evaluators," significantly enhancing our ability to "read" regulatory grammar, but are mostly limited to scoring existing sequences, making it difficult to actively explore the vast sequence space that has not yet been naturally sampled. This limitation is particularly pronounced in crop breeding contexts aimed at targeted improvement.

[0048] The emergence of denoised diffusion probability models (DDPMs) and genome language models in recent years has accelerated CRE research towards a generative design paradigm. Unlike traditional genetic algorithms, which are prone to getting trapped in local optima, generative models can "reconstruct" the potential distribution of natural sequences in a probabilistic sense by learning the manifold structure of natural sequences in high-dimensional space, thereby generating candidate sequences with both novelty and high biological fidelity. Existing research has shown that this approach has been able to produce synthetic elements with activity levels even superior to natural sequences in the fields of protein design and human regulatory genomics. This methodological shift from "evaluation" to "writing" allows researchers to bypass, to some extent, the lengthy and uncertain selection process of natural evolution, enabling them to directly construct sequences based on pre-defined regulatory properties, opening new technical channels for the rational design of various CREs, including enhancers. However, robustly transferring this approach to crop systems requires more detailed contextual testing and cross-material validation.

[0049] The root cause lies in the significant data scarcity and methodological lag in cis-regulation research on plants, especially cereal crops. Currently, mainstream plant research primarily relies on techniques such as RNA sequencing (RAN-seq), Chromatin Immunoprecipitation sequencing (ChIP-seq), and Assay for Transposase Accessible Chromatin with high-throughput sequencing (ATAC-seq) for descriptive annotation. While such data can indirectly infer regulatory potential through chromatin states, they struggle to provide the direct and quantitative sequence-activity mapping data required to support advanced deep learning models. Although several high-resolution ATAC-seq maps have been published in rice, quantitative STARR-seq data at the whole-genome scale remains relatively scarce. The limited existing data exhibits significant heterogeneity in library construction strategies, fragment length distribution, sequencing depth, and activity normalization standards, resulting in the lack of a unified enhancer strength benchmark set for cross-study comparison. Meanwhile, existing plant deep learning models are mostly focused on classification tasks, and the few works involving quantitative regression of activity are mostly localized attempts. Furthermore, there is a lack of integrated frameworks that embed genomic contexts and are geared towards generative design. Therefore, completing the "data-model-design" chain in crop systems, and moving from functional data to de novo enhancer design, has become a core scientific problem urgently needing breakthroughs, and also constitutes the basic premise and core purpose of this application.

[0050] Based on this, this application proposes the RED-T (Rice Enhancer Design via Transformer and Diffusion) integrated computational framework, aiming to achieve a closed loop between prediction and design. This application does not simply pile up algorithms, but follows this logical path: First, through a unified bioinformatics pipeline, multiple sets of rice STARR-seq data are reprocessed and conditionally normalized to construct a high signal-to-noise ratio benchmark dataset. Second, a pre-trained plant DNA language model is used to build the intensity predictor REAP, achieving accurate regression of enhancer sequence activity. On top of this, a diffusion generation model REDiff based on the U-Net architecture is introduced to generate diverse candidate sequences from random noise, forming a screening mechanism linked to the REAP score. Functional experimental evidence at the protoplast level shows that the synthetic enhancers generated and selected by this framework have significant biological activity. Overall, RED-T not only provides a practical paradigm for AI-driven crop enhancer design, but also lays a crucial methodological foundation for steadily extending "programmable cis-regulation" to major crops.

[0051] This application utilizes two publicly available rice (Oryza sativa ssp. japonica cv. Nipponbare) STARR-seq datasets to construct a standardized dataset for subsequent training of enhancer activity prediction models. The standard STARR-seq data is from the rice whole-genome enhancer map (NCBI GEO accession number GSE121231). The FACS-coupled STARR-seq (FACS-STARR-seq) data is from a systematic study of activating regulatory elements (AREs) in rice (NGDC BioProject number PRJCA011002). Both experiments were conducted on Nipponbare seedling protoplasts, each containing two independent biological replicates, and the reporter plasmid input library (DNA) and reporter gene cDNA library (RNA) were sequenced separately. This application downloaded raw paired-end sequencing data of all the above samples from GEO and NGDC for unified downstream analysis.

[0052] All STARR-seq raw reads underwent the same preprocessing, quality control, and genome alignment process. First, Trim Galore (v0.6.7, based on Cutadapt v4.0) was used to remove adapters and perform quality shearing on the reads, setting a base quality threshold Q≥30 and a minimum retained read length of 30 bp to remove low-quality and excessively short reads. After quality control, the reads were aligned to the rice Nipponbare reference genome IRGSP-1.0 using Bowtie2 (v2.4.5) in the --very-sensitive-local mode to allow for the retention of small amounts of vector sequence or redundant bases at both ends of the reads. Subsequently, SAMtools (v1.16) was used to filter the alignment results, retaining only properly paired (flag-f 2) primary alignments with an alignment quality score (MAPQ) ≥30 as high-confidence unique alignment records. Considering that the STARR-seq signal intensity depends largely on the number of times the inserted fragment is sampled, and that PCR duplicates are intentionally preserved in FACS-STARR-seq analysis to avoid underestimating activation intensity, this application does not remove duplicate alignments in subsequent analyses, but directly uses all high-confidence paired reads for fragment counting.

[0053] Enhancer activity quantification and candidate region screening were performed using the BasicSTARRseq (v1.2.0) package within the R statistical environment. This workflow uses a plasmid input library (DNA read count) as the baseline and a reporter cDNA library (RNA read count) as the output, counting the number of DNA and RNA reads on the genome at the insert fragment level. BasicSTARRseq uses a negative binomial distribution model to test for significant enrichment of RNA counts relative to DNA counts, defining the fragment-level log2(RNA / DNA) enrichment ratio as the enhancer activity intensity of that fragment (hereinafter referred to as log2FC). To minimize batch effects caused by differences in analytical methods, this application did not use the MACS2-based transcription peak and ARE definition workflow for FACS-STARR-seq data. Instead, starting from the original alignment results, activity quantification and significance assessment were completely re-performed according to the BasicSTARRseq pipeline, thereby establishing a unified statistical model and significance threshold between the two sets of data. The criteria for enhancing significance were: Bonferroni correction for multiple hypothesis testing, adjusted P-value < 0.001, and enrichment factor FC ≥ 1.3. To ensure good biological reproducibility, only fragments meeting the above thresholds in two independent biological replicates were retained.

[0054] For each STARR-seq experiment, after obtaining significant enhancer fragments in two repeats, this application used the merge command of BEDTools (v2.30.0) to merge overlapping (≥1 bp) fragments between the two repeats into a single non-redundant enhancer site. The activity intensity of this site was defined as the maximum log2FC in the two repeats, reflecting its highest possible transcriptional activation capacity under different experimental conditions. Subsequently, the enhancer sites obtained from the two experiments were further merged according to genomic coordinates. Regions located on the same chromosome and overlapping with each other were considered as the same enhancer site, and their log2FC values ​​in different experiments were recorded.

[0055] To construct a sequence input window of uniform length, this application uses the geometric center of each merged enhancer interval as a reference, extending 256 bp to both sides, and extracting a 512 bp genomic sequence as the standard input sequence for that enhancer site. After the above rigorous quality control, screening, and merging process, 13,414 non-redundant 512 bp enhancer sequences were finally obtained, which serve as the basic dataset for training the rice enhancer intensity prediction and generation model in this application.

[0056] This application constructs a deep learning model called REAP (Rice Enhancer Activity Predictor) for the quantitative prediction of rice enhancer activity. This model is based on a transfer learning strategy, using Plant DNABERT as the pre-trained backbone. Plant DNABERT is a Transformer model pre-trained on the genomes of 14 plant species. REAP is then fine-tuned on this basis to perform specific regression tasks. Transferring general knowledge from Plant DNABERT is crucial to addressing the challenge of the relatively limited size of the labeled dataset used in this application.

[0057] REAP employs a 6-mer tokenization strategy to process input sequences. The superiority of the 6-mer tokenization strategy has been proven in DNA language models. Compared to one-hot or BPE, 6-mer achieves the best balance between capturing core sequences of functional motifs, such as transcription factor binding sites (TFBS), and controlling vocabulary size. It encodes key functional motifs into indivisible words while effectively avoiding the contextual information loss problem of one-hot encoding. Therefore, in the process of this application, all DNA sequences are first centrally trimmed to 512 bp and then converted into overlapping 6-mer token sequences.

[0058] REAP employs a customized regression head instead of the standard [CLS] token pooling. This head first uses a learnable weighted layer-mixing strategy to process the hidden states (H) of the top K=4 Transformer Blocks of DNABERT. i ∈R L×D We perform a weighted summation on (D = 512) to obtain the fusion feature H. fused Next, H fused The data is fed into a multi-scale pooling module. This module computes and concatenates four pooling vectors in parallel: global average pooling, global max pooling, attention pooling, and pyramid pooling (window W∈{8, 16, 32, 64}, the result of which is then subjected to global max pooling). This step produces a concatenated high-dimensional feature vector V. pooled ∈R 7×D (3584 dimensions).

[0059] The REAP architecture also incorporates reverse complementarity (RC) equivariance. The model encodes the forward chain (Seq) in parallel during the forward pass. fwd ) and reverse complementary chain (Se qrc The characteristics of the two-chain multi-scale pooling vector V. pooled,fwd and V pooled,rc Averaged V before entering MLP final = 0.5×V pooled,fwd + V pooled,rc V final After passing through Layer Norm, the vector is fed into a two-layer MLP (3584→512→128, GELU activation). This MLP, acting as a Heteroscedastic Regression Head, contains two independent linear output layers that predict the mean μ and the log-variance log(σ), respectively. 2 ).

[0060] like Figure 1 As shown, the 512bp enhancer sequences, which are complementary in both forward and reverse directions, are segmented and embedded using 6-mer algorithms and then input into the Plant-DNABERT backbone with shared weights. Rank-16 LoRA adapters are inserted into the query, key, value, and dense linear layers of all Transformer Blocks in the backbone and participate in fine-tuning. Simultaneously, only the top two blocks in the original backbone weights are unfrozen, while the remaining blocks remain frozen. The hidden states of the forward and reverse directions are extracted from the top four blocks (L0-L3) as the input to the regression head. Figure 2 As shown, for each chain, the hidden states of the top four blocks are first subjected to learnable α-weighted hierarchical mixing, and then multi-scale features are obtained and concatenated through four parallel branches: Global Average Pooling (GAP), Global Max Pooling (GMP), Attention Pooling (ATTN), and Pyramid Pooling (PYR). The average of the positive and RC pooling features is used to obtain the RC-average features, which are then processed by LayerNorm and two MLP layers. The mean μ (log2FC) and log-variance logσ of the enhancer activity are output by two linear heads, respectively. 2 The algorithm is trained using heteroscedastic regression loss based on Gaussian NLL.

[0061] Model training and evaluation employ 5-fold cross-validation (5-Fold CV). To prevent sequences from appearing simultaneously in both the training and validation sets, which could lead to data leakage and overestimation of the model's generalization ability, the folds for cross-validation are generated using GroupKFold based on chromosome grouping.

[0062] This application employs a hybrid differential fine-tuning strategy to achieve efficient knowledge transfer and prevent overfitting on small datasets. Most of the original weights of the DNABERT backbone are frozen. Low-rank adaptation (LoRA) (R=16) is injected into the query, key, value, and dense layers of the backbone. Furthermore, all weights of the last two Transformer Blocks of DNABERT are unfrozen. The model is trained using the AdamW optimizer and differential learning rates. A custom regression head uses LR. Head = 1.0×10 −4 All parameters derived from the pre-trained model LR Base = 1.0×10 −5 The learning rate scheduler uses a cosine scheduler with a 10% warm-up. To prevent overfitting, this application also implements an early stopping strategy. During model training, the Pearson correlation coefficient on the validation set is monitored. If no improvement is achieved for 10 consecutive epochs, training will automatically terminate and the best-performing model weights will be loaded.

[0063] To specifically address the issues of uncertainty modeling and dynamic range compression in quantitative regression, this application designs a composite loss function LTotal, defined as:

[0064] L Total= L NLL + λ std L StdMatch + λ rc L RCConsist + λ reg L LogVarReg

[0065] Among them, the basic loss L NLL It is the Gaussian negative log-likelihood (NLL) loss, which utilizes the mean μ and variance σ predicted by the model. 2 (i.e., v), for overconfidence (σ) 2 Smaller samples that are mispredicted will be penalized more severely:

[0066]

[0067] L Std The scale-matching loss is used to address the dynamic range compression problem. It explicitly penalizes the difference between the standard deviations of the predicted and actual values ​​within a batch.

[0068]

[0069] L RCConsist It is the RC consistency loss, which acts as an auxiliary regularization term to reinforce biological priors through a computational path independent of the main architecture's RC averaging steps.

[0070] L RCConsist = MSE(μ fwd , μ rc )

[0071] L LogVarReg It is an L2 regularization term used to ensure the numerical stability of v during training:

[0072] L LogVarReg = mean(v 2 )

[0073] During the inference phase, two post-processing steps were applied to obtain the most robust prediction results. First, test-time augmentation was used to predict the forward and reverse complementary chains of the test sequence separately, and the average of the two predictions was taken as the raw prediction. Second, this raw prediction was calibrated using an ordination-preserving regression model. This calibrator is a nonparametric, monotonically increasing mapping function used to correct any remaining systematic nonlinear biases while maintaining the ordination of the predictions. It was trained on the validation set data of each fold cross-validation. The final test set performance is based on the ensemble prediction report of the 5-fold model.

[0074] This application evaluates the generalization performance of all models for enhancer activity regression on independent, rigorously reserved test sets. Evaluation metrics include the Pearson correlation coefficient (PCC) and Spearman rank correlation coefficient (SCC) to measure linearity and monotonic consistency, and the coefficient of determination (R²) to quantify regression accuracy and error. 2 The mean squared error (MSE) and variance of the four metrics were calculated. All models were trained on the same training, validation, and test sets, and model selection was performed based on performance on the validation set combined with an early stopping mechanism. The test set was used only for a one-time evaluation of the final model performance. To control for the impact of randomness, each model was trained independently five times under different random seeds, and the average performance and variance of the four metrics on the test set were calculated.

[0075] To systematically compare the REAP model with other mainstream models, this application compared representative plant DNA language models under the PDLLMs framework, such as Plant-DNAMamba, Plant-DNABERT, Plant-DNAGPT, Plant-DNAGemma, and Plant Nucleotide Transformer. To ensure absolute fairness in the comparison, pre-trained weights and tokenizers were obtained from the official code repositories of each model, and a regression head identical to that of REAP was connected above the output layer of each backbone network. All models were fine-tuned end-to-end using identical data partitioning, loss functions, and optimizers. Except for a small-scale search on the validation set for the learning rate and warm-up ratio, all other training configurations remained consistent. This setup ensures that the final comparison results primarily reflect the differences in the representational capabilities of the model backbones, rather than biases caused by differences in prediction head structures or training strategies.

[0076] In summary, the methods for constructing and training the enhancer activity prediction model REAP include:

[0077] Step 1: Sequence segmentation and backbone representation: The 512bp sequence is segmented using 6-mer and input into a pre-trained plant DNA language model (Transformer backbone) to extract contextual representation.

[0078] Step 2, Lightweight migration fine-tuning strategy: Introduce low-rank adaptation (LoRA) in the main attention module and fine-tune it together with partial layer unfreezing (e.g., unfreeze only a few top blocks + regression heads) to reduce data requirements and improve migration efficiency.

[0079] Step 3, Learnable layer mixing and multi-scale pooling regression head: Learnable weighted fusion of several hidden states at the top of the backbone is performed to obtain fused features; then the multi-scale pooling module is entered to compute and concatenate various pooling representations (such as global mean pooling / global max pooling / attention pooling / pyramid pooling, etc.) in parallel to obtain the global vector representation of the enhancer.

[0080] Step 4, Reverse Complementary (RC) Equivariant Inference: Encode the forward chain and the reverse complementary chain of the input sequence in parallel; fuse the two outputs before / after the regression head (e.g., take the average) to ensure RC equivariance and improve generalization stability.

[0081] Step 5, Composite Loss Training: Use heteroscedasticity loss based on Gaussian negative log-likelihood (Gaussian NLL), and can be superimposed with scale-matching loss, RC consistency regularization, and L2 regularization.

[0082] To achieve de novo design of rice enhancer sequences, this application constructs and trains a denoising diffusion probability model (Rice Enhancer Diffusion generator, REDiff). The core of this model framework is a specially customized U-Net network, which is trained to recover biologically meaningful enhancer sequences from pure Gaussian noise in an iterative denoising process.

[0083] Before model training and generation, all augmentation subsequences are normalized to a fixed length of L = 512 bp and characterized as a four-channel tensor with dimensions [C = 4, H = 512, W = 1], corresponding to the A / C / G / T channels respectively. To match the data distribution with the Gaussian noise model, its scale in the training space is linearly scaled to the interval [-1, 1]. The denoising network f... θ (x tThe network employs a U-Net structure tailored for nucleic acid sequences. Its encoding and decoding ends consist of alternating stacks of residual blocks and upsampling / downsampling layers, fused via skip connections. To strictly adapt to 1D sequence data, all backbone convolutional kernels are in (5, 1) form, ensuring that convolutions only aggregate local contextual information along the sequence length axis (H = 512). Normalization uses GroupNorm (32 groups), and the non-linear activation function is SiLU. To explicitly capture the long-range dependencies upon which the enhancer functions depend, we insert self-attention modules on the coarser two-level feature maps (attention_resolutions = [1,2]), where Q / K / V key-value pairs are obtained by (1, 1) linear projection. Conditional information at diffusion time t is injected into each residual block via sinusoidal positional embedding followed by two MLP layers. The model has a base number of 96 channels with channel ratios of [1, 2, 4, 8], and each resolution level contains two residual blocks. The final output header convolutional layer is zero-initialized to reduce instability in the early stages of training. This network design ensures controllable computational efficiency while taking into account both local motifs and long-range interactions across sites in the nucleic acid sequence.

[0084] like Figure 3 As shown, the natural 512-bp enhanced subsequence is one-hot encoded to obtain a 4×512 matrix. After linear scaling to [−1, 1], Gaussian noise is gradually added through a cosine-scheduled forward diffusion process to obtain x_ at different time steps. t .like Figure 4 As shown, at each diffusion time step, x_ t The input is a one-dimensional ResNet-like U-Net, consisting of four resolution levels (96 basic channels), symmetrical skip connections between the encoder and decoder, and self-attention and bottleneck modules inserted at a coarse scale. The diffusion step t is processed through sinusoidal positional encoding and two MLP layers to obtain temporal embeddings, which are then injected into all residual blocks. Finally, a zero-initialized 1×1 convolution is used to output noise to predict v_. t .like Figure 5 As shown, the sampling stage starts from Gaussian noise z~N(0, I), uses REDiff U-Net with EMA weights for multi-step inverse denoising to obtain x_0, then scales it back to [0, 1] and normalizes it to A / C / G / T probabilities by channel. Discrete 512-bp synthetic enhancers are generated through temperature sampling; the activity variance σ predicted by REAP is... 2This is used to adjust the noise level in backdiffusion, thereby prioritizing the exploration of sequence regions where the model is uncertain.

[0085] The forward diffusion process is defined as a fixed Markov chain that progressively adds noise to the real sequence x0 over T = 1000 discrete time steps. Let the noise intensity β... t α generated by cosine scheduling t = 1 − β t , At any time t, the noise sample can be represented as:

[0086] , , .

[0087] Inverse denoising process p θ (x t-1 |x t ) is used to approximate the true posterior q(x) t-1 |x t To improve numerical stability and convergence speed, a v-prediction-based parameterization is adopted, which is theoretically equivalent to the traditional ε-prediction. The network outputs v. θ (x t The relationship between , t), and noise reconstruction is as follows: , The model training loss is: To further improve training efficiency and generation fidelity, this application does not treat the loss at all time steps t equally, but instead employs minimum signal-to-noise ratio (min-SNR) weighting: , γ = 2.0. This strategy utilizes L for different t values. main The loss is weighted, which effectively reduces the loss contribution during low signal-to-noise ratio periods and instead prioritizes training during high signal-to-noise ratio periods.

[0088] After training, EMA weights are used for inference sampling. The new sequence is generated from standard Gaussian noise x. T Starting at ~N(0,I), a pre-trained EMA model is used to perform a reverse denoising process of T = 1000 steps. At each step t > 0, random noise is added using the posterior variance of the REAP to guide the sampling trajectory and increase the diversity of the generated data.

[0089] ,

[0090] The mean and variance are respectively:

[0091] , .

[0092] When t = 0, we get Then, it is linearly scaled back from pm1 space to [0, 1] ( Then, normalization is performed along the channel dimension to obtain a valid A / C / G / T probability distribution for each position. To achieve a balance between functional feasibility and diversity, temperature decoding is employed, and P(b)∝ for each position. T decode = 0.90, then perform independent polynomial sampling to obtain the final bases b∈{A, C, G, T}. In actual operation, sample in blocks in parallel until the target number is reached, and then export the generated sequence.

[0093] To quantitatively assess the extent to which the model-generated enhancers (GEs) reproduce the biological characteristics of natural rice enhancers (NEs) and simultaneously evaluate their sequence-level novelty, three complementary evaluation metrics were designed.

[0094] Jaccard distance: Jaccard distance measures the compositional difference between two sets. Let c be the k-mer counts of sequences A and B. A (ω), c B (ω), then:

[0095] .

[0096] This application divides the generated enhancer sequence into 5-mer short sequences that are continuous and overlapping with the natural enhancer sequence, forming two sets. The Jaccard distance between the two sequences is calculated using these two sets. The smaller the J value, the closer the word frequency distributions of the two sequences are, and the closer the model is to the natural distribution in terms of local motif composition.

[0097] Levenshtein distance: The Levenshtein distance is the minimum number of edit operations required to transform string A into B, including replacement, insertion, and deletion. It can be used to verify the novelty of the generated sequence. The larger the Levenshtein distance, the lower the similarity between the two sequences.

[0098] GC content: The GC content of each sequence in GEs and NEs was calculated and the statistical distribution of the two populations was compared to assess whether the model reproduced the genomic background composition specific to rice enhancers.

[0099] In summary, the methods for de novo enhancingr generation via REDiff denoising diffusion include:

[0100] Step 1, Sequence Continuous Representation: Perform one-hot encoding on the 512bp sequence and linearly scale it to a continuous space of [−1, 1] as the input for diffusion training.

[0101] Step 2, Definition of Forward Diffusion: Define a forward diffusing Markov chain q(x) at discrete time steps. t |x0), using a preset noise schedule (e.g., cosine schedule), the real sequence is gradually perturbed to approximately Gaussian noise.

[0102] Step 3: Training the denoising network: Construct a one-dimensional ResNet-style U-Net as the denoising network, predicting noise / velocity parameters at each time step (v-parameterization can be used); during training, a weighting strategy (such as min-SNR weighting) is used to improve numerical stability and convergence speed.

[0103] Step 4, Inverse Denoising Sampling: Using exponential moving average (EMA) weights from x T Starting from ~N(0, I), iteratively perform inverse denoising to obtain... .

[0104] In particular, the REAP-guided heteroscedastic uncertainty diffusion sampling introduces uncertainty noise in the back-diffusion process. At each backward time step t, the candidate sequence representation is first obtained from the current state, and then input into REAP to obtain the σ of that candidate sequence. 2 σ 2 The scale of random noise injection in this step is adjusted so that, for regions with high uncertainty, increasing the noise results in stronger exploration, while for regions with low uncertainty, decreasing the noise results in more stable convergence.

[0105] To achieve de novo design of rice enhancer regulatory activity, an end-to-end integrated computational framework called RED-T (RiceEnhancer Design by Transformer-Diffusion) was constructed. Based on uniformly organized high-quality rice STARR-seq data, RED-T combines advanced Transformer sequence models with generative diffusion models, forming two tightly coupled functional modules: the sequence-activity predictor REAP and the sequence generator REDiff. The former interprets the sequence syntax of rice enhancers at the genome scale, while the latter utilizes this grammatical knowledge for generative exploration, enabling direct advancement from raw functional genomic data to the de novo design and experimental validation of candidate enhancers.

[0106] In the first module, a sequence-activity prediction model, REAP (Rice Enhancer Activity Predictor), was established. This model takes a 512bp DNA sequence as input and can predict the corresponding STARR-seq activity based solely on the base sequence. REAP utilizes transfer learning based on the pre-trained Plant DNABERT to characterize the complex nonlinear dependencies within enhancers and extracts a high-dimensional regulatory vocabulary through attention mechanisms and deep representations. Simultaneously, a heteroscedastic regression head is introduced, enabling REAP to provide uncertainty assessments while outputting predicted activities. This effectively identifies regions with low prediction confidence, providing quantitative and controllable quality control for subsequent large-scale in silico screening.

[0107] In the second module, abandoning the traditional local perturbation search strategy, a denoising diffusion generator REDiff (Rice Enhancer Diffusion generator) based on the U-Net architecture was developed to perform sequence generation. Unlike traditional search strategies based on preset rules or genetic algorithms, REDiff learns the high-dimensional data distribution of natural enhancers and masters implicit cis-regulatory grammar. Starting from random Gaussian noise, it gradually generates a batch of diverse 512bp candidate enhancers with potential biological activity through a reverse denoising process. The generated sequences are then scored for high-throughput activity by REAP, forming an efficient "generation-evaluation-screening" closed loop. Thus, RED-T significantly expands the sequence search space while ensuring that the designed enhancers have both high predictive activity and good biological interpretability.

[0108] like Figure 6 As shown in Figure A, which is a schematic diagram of the REAP sequence-activity predictor, a single 512bp enhancer sequence is first encoded into 6-mer terms and input into a pre-trained Plant-DNABERT backbone network. After pooling and regression head processing, the enhancer activity prediction value is output. Figure B is a flowchart of enhancer generation and REAP scoring based on REDiff. The natural 512bp enhancer is one-hot encoded and scaled to the [−1, 1] space for training the REDiff diffusion U-Net model. During the sampling phase, the model starts from Gaussian noise, gradually denoises and decodes to generate synthetic enhancer sequences, which are then used for activity prediction via REAP. Synthetic sequences with high prediction scores are retained as candidate designs for subsequent experimental validation.

[0109] This application uses 13,414 high-quality rice enhancement sequences collected and organized, randomly divided into a training set and an independent test set at a ratio of 9:1, and uses these to train the REDiff model. Figure 7As shown, both the training set loss and the test set loss exhibit a synchronous decreasing trend, reaching a convergence plateau after approximately 500 epochs. The small generalization error between the two loss curves indicates that the model can effectively learn the data distribution of natural enhancers without overfitting.

[0110] Since the loss function only reflects pixel-level denoising capability, k-mer frequency correlation (R0) is introduced to further quantitatively evaluate the biological fidelity of the generated sequence. 2 As an evaluation metric, the k-mer frequency correlation between 500 randomly selected NEs from the test set and the overall training set was calculated first to establish an evaluation benchmark. The results showed that the natural sequences, serving as the control group, had higher k-mer, 5-mer, and 6-mer R-values. 2 The values ​​were 0.98, 0.97, and 0.93 respectively. Figure 8 As shown by the three colored dashed lines, this value represents the theoretical upper limit that the generative model should approximate under ideal conditions. Subsequently, during training, 500 generated sequences (GEs) are randomly sampled every 200 epochs, and their k-mer frequencies R with the training set are calculated. 2 .like Figure 8 As shown, with each generation of training, the k-mer frequency distribution of GEs gradually aligns with the natural sequence distribution. At Epoch 2400, the model performance reaches its optimum, with the 4-mer, 5-mer, and 6-mer R-values ​​of GEs being optimal. 2 The results showed that the values ​​were 0.96, 0.93, and 0.85, respectively. This indicates that the sequences generated by REDiff possess statistically comparable characteristics to natural enhancers in terms of local motif features. Based on this evaluation, this application selected the model parameters for Epoch 2400 for subsequent sequence generation tasks.

[0111] After validating REDiff's ability to reproduce local sequence motif features through k-mer frequency analysis, the biological fidelity and diversity of gene sequences (GEs) are further evaluated at the overall sequence level. To this end, three metrics are introduced: 5-mer-based Jaccard distance, Levenshtein edit distance, and GC content. Specifically, for each archive point in the training trajectory, 500 GEs are randomly sampled, and the minimum distance of each GE relative to all nullable sequences (NEs) in the training set is calculated to quantify the nearest neighbor proximity of the generated sequence to the real distribution in the sequence space.

[0112] like Figure 9 and 10As shown, with the advancement of training iterations, the Jaccard distance and Levenshtein distance between GEs and NEs generally show a decreasing trend, eventually converging into a relatively stable interval. In the early stages of training, the high median distance indicates that the generated sequences have not yet fully learned the sequence grammar of natural enhancers. However, around epochs 2000-2400, the distributions of both types of distances tend to plateau, suggesting that REDiff has pushed the generated sequences toward the characteristic manifold of natural enhancers. Notably, the converged Levenshtein distance distribution did not collapse to zero or extremely low values, and maintained a stable interquartile range, indicating that the model did not simply memorize or copy training samples to fit the data, but rather generated sequences with a certain degree of novelty at the base arrangement level.

[0113] Regarding genomic background characteristics, the GC content distribution of GEs fluctuated significantly in the early stages of training, but then quickly converged and stabilized within a range highly consistent with that of natural enhancers. Figure 11 This result corroborates the aforementioned k-mer analysis, indicating that the diffusion generator can effectively capture and maintain the nucleotide composition preference unique to the rice genome, avoiding statistical drift to non-physiological regions during the generation process.

[0114] Figure 7 In the figure, REDiff shows the change in mean squared error (MSE) loss of the training and test sets over 3000 training epochs. Figure 8 In the figure, the determination coefficient R2 of the k-mer frequency distribution (k = 4, 5, 6) between generated enhancers (GEs) and natural enhancers (NEs) varies with the training rounds, and the performance of the training set and the test set are given respectively. Figure 9-11 The similarity of GEs at the global sequence level and the composition of GC under different training rounds. Figure 9 Let be the minimum Jaccard distance distribution between GEs and the training set NEs based on 5-mer. Figure 10 The minimum Levenshtein distance distribution between GEs and NEs is given (normalized to 512bp). Figure 11 The GC content distribution of GEs is shown. 500 GEs were randomly sampled in each archived training round to calculate the above indicators, and their overall trend was displayed using a box plot.

[0115] Based on the above multidimensional indicators, REDiff, during training, gradually shortens the nearest neighbor distance between the generated space and the native space, reproducing the main statistical characteristics of real enhancers. On the other hand, it also preserves significant endogenous diversity at the base level. This indicates that, at the selected optimal training epoch (2400 epochs), the model has the ability to generate candidate enhancers that possess both biological plausibility and sequence innovation.

[0116] To provide a reliable sequence-activity scorer within the RED-T framework, a regression model REAP (Rice Enhancer Activity Predictor) was constructed based on 13,414 uniformly organized rice STARR-seq enhancer sequences. REAP uses a 512bp DNA fragment as the sole input and, based on a pre-trained Plant-DNABERT backbone and a customized regression head strategy, achieves quantitative prediction of enhancer activity.

[0117] While maintaining complete consistency in network structure, training process, and data partitioning, the impact of three DNA segmentation methods on Plant-DNABERT was first examined: 6-mer, BPE, and single-base encoding. All three variants were trained independently within a 5-fold cross-validation framework grouped by chromosome, with the Pearson correlation coefficient (PCC) on the validation set serving as the primary selection metric. Figure 12 As shown.

[0118] The results showed that all three segmentation methods could reproduce the experimentally measured enhancer activity gradients, but there were systematic differences in prediction accuracy. The PCC of 6-mer encoded Plant-DNABERT on the 5-fold validation set was 0.72 ± 0.01, significantly higher than BPE (0.69) and single-base encoding (0.70). This difference remained stable across the five folds, indicating that 6-mer segmentation can more effectively utilize limited training samples to learn enhancer sequence syntax. This advantage is highly consistent with biological intuition; the scale of 6-mer fragments is similar to that of typical transcription factor binding motifs, which is beneficial for the model to directly capture local regulatory motifs and their combination rules. In contrast, BPE encoding may disrupt functional motif boundaries under current data scales, while single-base encoding requires a longer context to recover the equivalent k-mer statistical pattern.

[0119] Following the establishment of 6-mer as the optimal encoding strategy for REAP, a systematic benchmark test was conducted against the most representative plant genome deep learning models within a unified evaluation framework. This application constructed an evaluation matrix containing 15 model variants: five advanced pre-trained models—Plant-DNABERT, Plant-DNAMamba, Plant-DNAGemma, Plant-DNAGPT, and Plant Nucleotide Transformer (NT)—were selected as backbone networks, each adapted to 6-mer, BPE, and single-base sequence encoding methods. To ensure rigorous comparison, all models were uniformly configured with a 512bp input window and the same regression head structure, and end-to-end fine-tuning was performed under completely consistent five-fold cross-validation and independent test set partitioning.

[0120] The evaluation results based on five-fold cross-validation show that, Figure 13 As shown, the Plant-DNABERT (REAP) model using 6mer encoding performs exceptionally well in explaining regression variance and controlling prediction error. Specifically, the R² of REAP is significantly higher than that of other models. 2 The R² value reached 0.50, significantly exceeding the 0.40-0.46 range predominantly found in other model combinations. Simultaneously, its MSE was the lowest (0.15), better than the 0.16–0.20 range of other models. In terms of correlation metrics (PCC and SCC), REAP also ranked first among all combinations, indicating that its prediction results maintain high accuracy while best preserving the relative ordering among samples. Results on the independent test set further confirmed the robustness of this conclusion; REAP's PCC remained stable at around 0.70, and R²... 2 The coefficient of performance remained at 0.49, with an MSE of approximately 0.14. In contrast, the suboptimal model combinations (such as NT-BPE or Mamba-single base) showed a smaller difference from REAP in PCC, but in R... 2 The average performance was generally 0.04-0.06 behind, and the MSE was significantly higher, which highlights the stronger generalization ability of REAP when dealing with unseen data.

[0121] Furthermore, different pre-trained architectures exhibited varying adaptive preferences for sequence encoding methods. Plant-DNABERT consistently demonstrated strong adaptability to 6-mer encoding, outperforming BPE and single-base encoding across all metrics. While architectures like Mamba, Gemma, and GPT showed slight improvements in PCC when paired with single-base encoding, the NT model performed better when using BPE. However, even when comparing the performance of each backbone network under optimal encoding fit, REAP (BERT-6mer) maintained a clear advantage in key metrics measuring model fit.

[0122] To quantitatively evaluate the contribution of different design modules to the predictive performance of REAP, a systematic ablation analysis was conducted on the loss function and fine-tuning strategy while maintaining complete consistency in data partitioning and backbone structure. Figure 14As shown in the figure, each bar represents a model configuration, and the vertical axis represents the PCC obtained in five-fold cross-validation grouped by chromosome. The design matrix below uses "● / ×" to indicate the enabling status of different components in each model configuration, where ● indicates enabled and × indicates disabled. The four rows correspond to heteroscedastic NLL loss (Loss type), correlation regularization term (Corrreg.), LoRA fine-tuning, and whether the backbone Transformer blocks are unfrozen (Backbone unfreezing).

[0123] Regarding the loss function, while keeping the LoRA and backbone fine-tuning strategies unchanged, we compared three settings: the traditional Huber loss, pure heteroscedasticity NLL, and heteroscedasticity NLL combined with correlation regularization. The full configuration achieved a PCC of approximately 0.72 on the validation set, significantly higher than NLL without regularization and the Huber version, and the differences among the three were consistent across different folds. The results indicate that explicitly incorporating correlation regularization into the loss effectively strengthens the monotonic relationship between predicted values ​​and experimentally measured activity, thereby improving the overall fit without sacrificing stability.

[0124] Regarding fine-tuning strategies, the optimal loss was further fixed, and various combinations were compared, including head-only, LoRA only, only thawing the last two Transformer blocks, and LoRA + block thawing. It can be seen that in settings where only the regression head is fine-tuned or only the backbone terminal blocks are thawing, the PCC remains around 0.68. Adding LoRA with the backbone completely frozen slightly improves the PCC to approximately 0.69, but the increase is limited. The model performance reaches its peak only when LoRA and backbone block thawing are enabled simultaneously. This indicates that, while maintaining a relatively controllable overall parameter count, locally thawing the high-level backbone and performing low-rank adaptation via LoRA is key to fully exploiting the characterization potential of Plant-DNABERT and improving sequence-activity mapping accuracy.

[0125] Figure 12 When using 6-mer, BPE, and single-base encoding methods, the prediction results of REAP in five-fold cross-validation grouped by chromosome showed that 6-mer encoding was generally better than BPE and single-base encoding in terms of correlation and fit. Figure 13 The results show the benchmark test results of REAP and various plant DNA language models under different word segmentation strategies. The results indicate that REAP using 6-mer encoding performs best overall among all backbones. Figure 13This is an ablation analysis of different combinations of loss functions and fine-tuning strategies for REAP. The bars represent the five-fold cross-validation (PCC). The design matrix below uses "● / ×" to indicate whether each configuration enables the corresponding component, where ● indicates enabled and × indicates disabled. The four rows correspond to heteroscedastic NLL loss (Losstype), correlation regularization (Corr reg.), LoRA fine-tuning, and backbone Transformerblocks unfreezing, respectively. The results show that REAP achieves optimal predictive performance only when heteroscedastic NLL + correlation regularization are used simultaneously, combined with LoRA and backbone local unfreezing.

[0126] In the field of crop genomics, the long-standing research paradigm has primarily focused on "reading" the cis-regulatory code, i.e., using deep learning models to predict the activity of natural sequences. However, for precision breeding and synthetic biology, simply "discriminating" the quality of sequences is far from sufficient; the core challenge lies in how to "generate" novel sequences with specific functions. The RED-T framework proposed in this application, by integrating the semantic understanding capabilities of pre-trained Transformers with the generative capabilities of diffusion models, achieves a shift from discriminative to generative approaches in rice enhancer research. Given the enormous potential of deep learning in de novo enhancer design in model organisms, this application extends this technological system to major crops, providing a new technical path and theoretical basis for crop synthetic biology and precision molecular breeding.

[0127] The core challenge in synthetic enhancer design lies not in how to "assemble" a number of highly active sequences within the astronomically large combinatorial space of 4^n, but in how to explore efficiently and directionally under the constraint of limited experimental throughput. Early works based on DeepSTARR mostly employed strategies similar to random selection. This approach has broad coverage but hardly utilizes the model's internal understanding of regulatory grammar, resulting in extremely low efficiency. Recent breakthroughs, such as the DREAM framework which combines deep learning predictors with genetic algorithms, have achieved enhancer optimization in sequence space similar to directed evolution. In contrast, the denoising diffusion probability model used in RED-T represents a qualitative leap in sequence design methodology. The REDiff generation model does not blindly search or locally patch in the sequence space, but learns the high-dimensional probability distribution manifold of natural enhancers in the latent space. Analysis shows that REDiff-generated sequences, while preserving key biological grammar, exhibit significant differences from natural sequences in Levenshtein edit distance. This indicates that the model has successfully explored the dark matter region of sequences that natural evolution has not yet touched. This ability to reconstruct biological meaning from pure noise gives RED-T a significant advantage over traditional evolutionary algorithms in terms of the novelty and diversity of the generated sequences.

[0128] In particular, computationally designed synthetic enhancers significantly outperform the strongest natural enhancers in the rice genome in terms of activity. Natural enhancers not only need to drive expression but also avoid metabolic burden or gene network imbalance caused by overactivation, while being constrained by genomic location, mutation rate, and population history. In contrast, the in silico evolutionary process of RED-T removes these complex biological constraints, focusing solely on "maximizing activity." By optimizing motif affinity, copy number, and spatial arrangement, RED-T is essentially exploring the physicochemical limits of cis-regulatory elements. This demonstrates that the regulatory potential of plant genomes is far from being exhausted by natural sequences, and that artificial design can fully unleash the regulatory capabilities "sealed" by evolution.

[0129] The above specific embodiments are merely several preferred embodiments of the present invention. Based on the technical solutions of the present invention and the relevant teachings of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A method for predicting and generatively designing enhancer strength in rice, characterized in that, include: First, a benchmark dataset was constructed by reprocessing and condition-specific normalization of multiple sets of rice STARR-seq data through a unified bioinformatics pipeline. Secondly, a rice enhancer activity predictor was built using a pre-trained plant DNA language model, which enabled the regression of enhancer sequence activity. Building upon this, a rice enhancer diffusion generator based on the U-Net architecture is introduced to generate diverse candidate sequences from random noise and form a screening mechanism linked with the rice enhancer activity predictor score. The steps for constructing and training the rice enhancer activity predictor include sequence segmentation and backbone representation, lightweight transfer fine-tuning strategy, learnable layer mixing and multi-scale pooling regression head, reverse complementary equivariant inference, and composite loss training; the steps for the denoising diffusion of the rice enhancer diffusion generator to generate enhancers from scratch include sequence continuous representation, forward diffusion definition, denoising network training, and reverse denoising sampling. Among them, the rice enhancer activity predictor guides heteroscedastic uncertainty diffusion sampling, introduces uncertainty noise in the back diffusion, and at each back time step, the candidate sequence characterization is first obtained from the current state, input into the rice enhancer activity predictor to obtain the variance of the candidate, and the variance is used to adjust the scale of random noise injection in this step.

2. The method for predicting and generatively designing enhancer strength in rice according to claim 1, characterized in that: In the sequence segmentation and backbone representation steps, the 512 bp sequence was segmented using 6-mer and input into a pre-trained plant DNA language model to extract contextual representation.

3. The method for predicting and generatively designing enhancer strength in rice according to claim 1, characterized in that: In the lightweight migration fine-tuning strategy, low-rank adaptation is introduced into the backbone attention module and fine-tuned in conjunction with partial layer unfreezing.

4. The method for predicting and generatively designing enhancer strength in rice according to claim 1, characterized in that: In the steps of learnable layer mixing and multi-scale pooling regression head, the hidden states of several layers at the top of the backbone are fused using learnable weighted fusion to obtain fused features, which are then entered into the multi-scale pooling module to compute and concatenate multiple pooling representations in parallel to obtain the global vector representation of the enhancer.

5. The method for predicting and generatively designing enhancer strength in rice according to claim 1, characterized in that: In the reverse complementary equivariance inference step, the forward chain and the reverse complementary chain of the input sequence are encoded in parallel, and the two outputs are fused before and after the regression head to ensure the reverse complementary equivariance.

6. The method for predicting and generatively designing enhancer strength in rice according to claim 1, characterized in that: In the training step of composite loss, heteroscedasticity loss based on Gaussian negative log-likelihood is adopted, and scale matching loss, consistency regularization, and L2 regularization can be superimposed.

7. The method for predicting and generatively designing enhancer strength in rice according to claim 1, characterized in that: In the step of sequence continuous representation, the 512 bp sequence is one-hot encoded and linearly scaled to a continuous space of [−1,1] as the diffusion training input.

8. The method for predicting and generatively designing enhancer strength in rice according to claim 1, characterized in that: In the forward diffusion definition step, a forward diffusion Markov chain with discrete time steps is defined, and a preset noise schedule is used to gradually perturb the real sequence to approximately Gaussian noise.

9. The method for predicting and generatively designing enhancer strength in rice according to claim 1, characterized in that: In the training steps of the denoising network, a one-dimensional ResNet-style U-Net is constructed as the denoising network to predict noise and velocity parameters at each time step. During training, a weighted strategy is adopted to improve numerical stability and convergence speed.

10. The method for predicting and generatively designing enhancer strength in rice according to claim 1, characterized in that: In the reverse denoising sampling step, reverse denoising is performed iteratively starting from standard Gaussian noise using exponential moving average weights.

Images