A framework and system for de novo design of functional nucleic acids based on nucleic acid language models

Abstract

The application discloses a method for constructing a nucleic acid de novo design model based on a nucleic acid language model, which comprises the following steps: using a nucleic acid language model fine-tuned by SELEX data to encode input data, converting the input data into a high-dimensional low-bias hidden space, and converting the hidden space into a low-dimensional low-noise latent space vector by using a linear layer; screening the latent space vector by using an E-HEBO algorithm and affinity prior data, so that an optimal affinity latent space vector is obtained; further, decoding the optimal affinity latent space vector by using a linear layer and a self-attention layer, so that a candidate sequence is generated. In addition, the method further comprises iterative encoding and decoding, and iterative generation of affinity prior data, so that a high-quality, high-accuracy and high-activity candidate sequence is obtained. The application provides an efficient artificial intelligence method for nucleic acid de novo design, and is helpful to the design and screening of functional nucleic acids and the application of the functional nucleic acids in biological medicines.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, nucleic acid screening, and biomedicine. Specifically, it relates to the use of artificial intelligence technology, particularly based on nucleic acid language models, for the design, optimization, and screening of nucleic acids, and consequently, methods for the research and design of nucleic acid drugs. Background Technology

[0002] Nucleic acid aptamers are a class of highly specific and affinity functional nucleic acid molecules that can precisely recognize and bind to target molecules, such as cell surface proteins, cytokines, and small molecules. Through stable binding to their targets, aptamers can regulate biological functions, demonstrating broad application potential. Thanks to their unique molecular recognition capabilities, nucleic acid aptamers have been widely used in disease diagnosis, targeted drug delivery, molecular imaging, and biosensors. In precision medicine, aptamers play a particularly important role, enabling targeted therapy against individualized biomarkers, thereby promoting personalized medicine and bringing new breakthroughs and innovations to drug development and disease detection.

[0003] SELEX (Systematic Evolution of Ligands by Exponential Enrichment) is a general and powerful method for screening nucleic acid aptamers. Through iterative selection and amplification processes, it can screen for nucleic acid aptamers that can specifically bind to specific targets. The main SELEX workflow includes: preparing a random nucleic acid library, binding the nucleic acid library to the target molecule, separating bound and unbound sequences, amplifying the bound sequences (PCR amplification), repeating the screening, and selecting the optimal sequence. However, due to factors such as the nature of the target protein, PCR amplification bias, and library incompleteness, the binding ability of sequences screened frequently by SELEX may not be very strong, which can sometimes lead to SELEX experiments failing. For failed SELEX experiments, each round generates a large amount of sequencing data, which contains rich information about the relationship between the nucleic acid sequence and the target substance.

[0004] Initially, research on SELEX sequencing data focused on using bioinformatics algorithms to score sequencing sequences based on sequence features and frequency of occurrence in each round of sequencing, identifying low-frequency but potentially highly binding nucleic acid sequences, such as RaptRanker. Another research approach involves generating entirely new sequences based on the distribution of sequencing data, known as de novo aptamer design, such as RaptGen. RaptGen was the first to propose using SELEX sequencing data to generate highly binding sequences not found in nucleic acid libraries. It utilizes deep learning methods to learn the latent distribution of sequencing data, employs a VAE architecture, and uses Bayesian optimization in the hidden space to generate highly binding nucleic acid sequences. However, RaptGen's encoder is based on a CNN, which is less effective in sequence modeling compared to self-attention mechanisms. RaptGen's hidden space has low dimensionality; each sequence consists of only two-dimensional vectors, failing to fully represent the semantics of each sequence, thus affecting generation performance. Furthermore, RaptGen's decoder is constructed using a Hidden Markov Model, resulting in significant time complexity for inference and limiting the speed of sequence generation.

[0005] Therefore, there is an urgent need in this field for a method and system for constructing a de novo design model of SELEX sequences with low noise and high search efficiency, so as to generate a large number of high-quality and high-accuracy sequences, and further screening based on these sequences. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for constructing de novo design and screening models of nucleic acid SELEX sequences with high efficiency and high accuracy.

[0007] A first aspect of the present invention provides a method for constructing a nucleic acid de novo design model based on a nucleic acid language model, the method comprising the steps of:

[0008] (S1) Provide a nucleic acid dataset, which includes a SELEX dataset; the SELEX dataset includes affinity nucleic acid sequences and / or affinity-free nucleic acid sequences; input the nucleic acid dataset into an encoder;

[0009] (S2) In the encoder,

[0010] (S2.1) Use a word segmenter to convert the nucleic acid sequences in the nucleic acid dataset into multiple tokens to obtain preprocessed nucleic acid sequences;

[0011] (S2.2) The preprocessed nucleic acid sequence is converted into a high-dimensional latent space vector using a nucleic acid language model; the high-dimensional latent space vector is reduced to an N-dimensional latent space vector using a linear layer, where N is an integer from 7 to 9;

[0012] (S3) The N-dimensional latent space vectors are filtered through the following steps:

[0013] (S3.1) Use a clustering algorithm to cluster all the N-dimensional latent space vectors obtained in step (S2), and select the central latent space vector of each cluster;

[0014] (S3.2) Using the latent space vector of the affinity nucleic acid sequence as prior data, a Bayesian optimization algorithm is used to search for latent space vectors with higher affinity than the prior data within a preset range; and / or

[0015] (S3.3) Using the latent space vector of the affinity nucleic acid sequence as prior data, select a latent space vector that is close to the latent space vector of the affinity nucleic acid sequence in distance;

[0016] Input the latent space vectors generated in steps (S3.1), (S3.2) and / or (S3.3) into the decoder;

[0017] (S4) In the decoder, the latent space vector is mapped to a probability distribution corresponding to each symbol using a network model, in order to generate candidate sequences and the predicted probabilities of each symbol.

[0018] (S5) The nucleic acid de novo design model based on the nucleic acid language model is formed by iterative optimization through the following steps:

[0019] (S5.1) Sequence optimization iteration: Multiple probability distribution-based sampling is performed on each symbol output from step (S4) to obtain multiple candidate sequences; the nucleic acid language model is used to score the multiple candidate sequences, and the sequence with the highest score is selected as the final sequence; and / or

[0020] (S5.2) Prior data iteration:

[0021] (S5.2a) Detect the affinity of the candidate sequences and divide the candidate sequences into affinity candidate sequences and non-affinity candidate sequences;

[0022] (S5.2b) Using the result of step (S5.2a) as the nucleic acid dataset, iterate steps (S2)-(S4) and (S5.2a) multiple times. During iteration, add the latent space vector of the affinity candidate sequence to the prior data in step (S3.2) and / or (S3.3) to obtain the final sequence.

[0023] In another preferred embodiment, the nucleic acid sequence is selected from the group consisting of DNA, RNA, or combinations thereof.

[0024] In another preferred embodiment, before step (S1), the step of training the nucleic acid language model and the decoder in the encoder is also included.

[0025] In another preferred embodiment, the nucleic acid language model and decoder in the encoder are trained through the following steps:

[0026] (a) Provide SELEX data for model training, wherein the SELEX data is the last round of SELEX sequencing data;

[0027] (b) The SELEX data is masked with a preset masking rate, and the nucleic acid language model is continuously pre-trained on the masked SELEX data to obtain the continuously pre-trained nucleic acid language model; wherein the preset masking rate is 10%-20%.

[0028] (c) Keeping the training result obtained in step (b) unchanged, train the decoder on the SELEX data to obtain a pre-trained decoder.

[0029] In another preferred embodiment, the nucleic acid language model includes: a DNA language model and an RNA language model.

[0030] In another preferred embodiment, the DNA language model includes: DNABERT, Evo.

[0031] In another preferred embodiment, the DNABERT is selected from the group consisting of DNABERT-3mers, DNABERT-4mers, DNABERT-5mers, or DNABERT-6mers; the RNA language model includes RNAErnie; and N = 8.

[0032] In another preferred embodiment, the DNABERT is DNABERT-3mers.

[0033] In another preferred embodiment, the RNA language model includes: RNAErnie.

[0034] In another preferred embodiment, the mask rate is 15%.

[0035] In another preferred embodiment, N = 8.

[0036] In another preferred embodiment, the clustering algorithm includes Gaussian mixture clustering.

[0037] In another preferred embodiment, the Bayesian optimization algorithm is selected from the group consisting of: the E-HEBO algorithm, the HEBO algorithm, or other Bayesian optimization-based algorithms.

[0038] In another preferred embodiment, the Bayesian algorithm is the E-HEBO algorithm.

[0039] In another preferred embodiment, step (S3.2) specifically includes the following steps: selecting a preset range based on the affinity of the prior data; when the affinity of the prior data is high, the range is ±10 for each dimension of the latent space vector; when the affinity of the prior data is low, the range is ±40 for each dimension of the latent space vector; and using the E-HEBO algorithm to search for latent space vectors with affinity superior to the prior data within the range.

[0040] In another preferred embodiment, "low affinity" means a binding specificity score <10, such as <5 or <0.

[0041] In another preferred embodiment, "high affinity" means a binding specificity score >10, such as >15 or >20.

[0042] In another preferred embodiment, in step (S3.3), the distance includes: Euclidean distance, Manhattan distance, and cosine distance.

[0043] In another preferred embodiment, in step (S3.3), the number of hidden space vectors that are close in distance to the hidden space vector of the affinity nucleic acid sequence is ≥1, ≥10, ≥20, ≥30, ≥40, ≥50, ≥60, ≥70, ≥80, ≥90, ≥100, ≥150, such as 100.

[0044] In another preferred embodiment, the network model includes a self-attention layer, a multilayer perceptron (MLP), a linear layer and a convolutional neural network (CNN), a recurrent neural network (RNN), Mamba, Hyena, GAN, or a combination thereof.

[0045] In another preferred embodiment, the network model is a self-attention layer.

[0046] In another preferred embodiment, the symbols include special symbols.

[0047] In another preferred embodiment, the special symbols include a start symbol (CLS), a stop symbol (EOS), and a sequence padding symbol (PAD).

[0048] In another preferred embodiment, step (S4) specifically includes the following steps:

[0049] (S4.1) The latent space vector is increased in dimension using a linear layer to obtain an increased-dimensional latent space vector; the increased-dimensional latent space vector is then input into a multi-head self-attention layer.

[0050] (S4.2) Perform the following steps in each subspace of the multi-head self-attention layer:

[0051] (S4.2a) Transform the increased-dimensional latent space vector into a query matrix, a key matrix, and a value matrix;

[0052] (S4.2b) Calculate the similarity score matrix based on the query matrix and the key matrix;

[0053] (S4.2c) Normalize the similarity score matrix using an activation function to obtain the attention weight matrix;

[0054] (S4.2d) Multiply the attention weight matrix and the value matrix to obtain the self-attention output of each subspace;

[0055] (S4.3) The self-attention outputs of each subspace are concatenated and mapped through a linear layer to generate a probability distribution corresponding to each symbol.

[0056] In another preferred embodiment, step (S5.1) specifically includes the following steps:

[0057] (i) Mask the special symbol with the lowest predicted probability in the candidate sequence, and set the first symbol and the last symbol of the candidate sequence as the start symbol and the end symbol, respectively;

[0058] (ii) The mask position is predicted by the result of step (i) through steps (S2.2) and (S4). During the prediction process, the prediction probability of the special symbol is masked to obtain the new symbol of the mask position and its prediction probability.

[0059] (iii) Replace the symbol at the corresponding mask position in the candidate sequence with the new symbol to obtain a new candidate sequence;

[0060] (iv) Mask the symbol with the lowest predicted probability in the new candidate sequence;

[0061] (v) Iterate through steps (ii)-(iv) to obtain the final sequence.

[0062] In another preferred embodiment, in step (S5.1), the number of iterations is 3.

[0063] In another preferred embodiment, before step (S5.2a), the step further includes: predicting the secondary structure of the candidate sequence using a structure prediction model and selecting the candidate sequence with the lowest Gibbs energy.

[0064] In another preferred embodiment, the structure prediction model includes: Mfold and SnapGene.

[0065] In another preferred embodiment, step (S5.2a) further includes the step of: detecting the affinity of the most frequent nucleic acid sequences in the SELEX dataset, and dividing the nucleic acid sequences into affinity candidate sequences and non-affinity candidate sequences.

[0066] In another preferred embodiment, the number of high-frequency nucleic acid sequences in the SELEX dataset is ≤100, such as ≤50, ≤20, or ≤10.

[0067] A second aspect of the present invention provides a method for generating highly active nucleic acid aptamers, the method comprising the steps of:

[0068] (X1) Provide a nucleic acid dataset, which includes a SELEX dataset; the SELEX dataset includes affinity nucleic acid sequences and / or affinity-free nucleic acid sequences; input the nucleic acid dataset into an encoder;

[0069] (X2) In the encoder, the nucleic acid sequences in the nucleic acid dataset are converted into multiple tokens using a word segmenter to obtain preprocessed nucleic acid sequences; the preprocessed nucleic acid sequences are converted into high-dimensional latent space vectors using a continuously pre-trained nucleic acid language model; and the high-dimensional latent space vectors are reduced to N-dimensional latent space vectors using a linear layer, where N is an integer from 7 to 9.

[0070] (X3) Cluster all the N-dimensional latent space vectors obtained in step (X2) using a clustering algorithm, select the center latent space vector of each cluster and input it into the decoder to generate candidate sequences; input the candidate sequences into the encoder for iterative optimization to obtain optimized sequences;

[0071] (X4) Using the latent space vector of a nucleic acid sequence with known affinity as prior data, the latent space vector closest to the prior data is determined based on spatial distance and input into the decoder to determine the corresponding nucleic acid sequence; the nucleic acid sequence is screened based on its properties, and the nucleic acid sequence whose properties reach a preset level is selected as the preferred sequence;

[0072] (X5) Detect the affinity of the top 10-100 nucleic acid sequences in the SEXEL dataset, the optimized sequence obtained in step (X3), and the preferred sequence obtained in step (X4), and classify them into affinity sequences and non-affinity sequences;

[0073] (X6) Input the nucleic acid sequences obtained in (X5) into the encoder to obtain the latent space vector of each sequence; using the nucleic acid sequences with known affinity and the latent space vectors of the affinity sequences as prior data, use the Bayesian algorithm to search for latent space vectors that are close to the prior data within a preset range, and input them into the decoder to generate candidate sequences; input the candidate sequences into the encoder for iterative optimization to obtain optimized sequences; filter according to the properties of the optimized sequences, and select the optimized sequences whose properties reach a preset level as the test sequences;

[0074] (X7) Detect the affinity of the test sequence and divide the test sequence into affinity test sequences and non-affinity test sequences;

[0075] (X8) Iterate through (X6)-(X7) multiple times. During the iteration, the latent space vector of the affinity test sequence is added to the prior data in step (X6) to obtain a highly active nucleic acid aptamer.

[0076] In another preferred embodiment, before (X1), the step (X0) is further included: training the nucleic acid language model and decoder in the encoder, including the steps:

[0077] (X0.1) Provides SELEX data for model training, which is the last round of SELEX sequencing data;

[0078] (X0.2) The SELEX data is masked with a preset masking rate, and the nucleic acid language model is continuously pre-trained on the masked SELEX data to obtain the continuously pre-trained nucleic acid language model; wherein the preset masking rate is 10%-20%;

[0079] (X0.3) Keeping the training result obtained in step (X0.2) unchanged, the decoder is trained on the SELEX data to obtain the trained decoder.

[0080] In another preferred embodiment, the mask rate is 15%.

[0081] In another preferred embodiment, N = 8.

[0082] In another preferred embodiment, the spatial distance is an Euclidean distance.

[0083] In another preferred embodiment, the property includes the Gibbs energy of the secondary structure.

[0084] In another preferred embodiment, “selecting the property to reach a preset level” means that the Gibbs energy is minimized.

[0085] In another preferred embodiment, the software for obtaining the Gibbs energy of the secondary structure of the nuclear sequence includes Mfold.

[0086] A third aspect of the present invention provides a de novo nucleic acid design system based on a nucleic acid language model, the system comprising:

[0087] An input unit is configured to input data, the data being a SELEX dataset, the SELEX dataset comprising affinity nucleic acid sequences and / or affinityless nucleic acid sequences;

[0088] The design unit is configured as a nucleic acid de novo design model based on a nucleic acid language model. The model generates highly active nucleic acid aptamers based on the SELEX dataset, thereby outputting the results of the design unit; wherein the model is constructed by the method described in the first aspect of the present invention.

[0089] An output unit is configured to output the results of the design unit.

[0090] A fourth aspect of the present invention provides an electronic device including a processor and a memory, the memory having a plurality of executable instructions, the processor being configured to read the instructions and execute steps as described in the method of the second aspect of the present invention.

[0091] A fifth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions that, when read and executed by a processor, implement the steps of the method as described in the second aspect of the present invention.

[0092] A sixth aspect of the present invention provides a computer program product comprising computer-executable instructions which, when executed by a processor, implement the steps of the method described in the second aspect of the present invention.

[0093] It should be understood that, within the scope of this invention, the above-described technical features of this invention and the technical features specifically described below (such as in the embodiments) can be combined with each other to form new or preferred technical solutions. Due to space limitations, they will not be described in detail here. Attached Figure Description

[0094] Figure 1 The flowchart of the model of this invention is shown. Here, Input embedding refers to the input embedding; Transformer refers to the Transformer framework; Latent embedding refers to the latent space embedding; and Self-AttentionBlock refers to the self-attention module.

[0095] Figure 2The experimental results of the present invention and RaptGen on different datasets demonstrate the semantic representation capabilities of the encoder.

[0096] Figure 3 The correlation between the kmers features of sequences generated by the present invention and the RaptGen decoder on different SELEX datasets and natural SELEX sequences is shown. RG represents RaptGen, and Our represents the present invention.

[0097] Figure 4 The diagram shows a comparison of the E-HEBO and HEBO algorithms using different active sequences as starting points in the hidden space of this invention. The horizontal axis represents different iteration rounds, and the three figures from top to bottom represent the results of optimization using different active sequences as prior knowledge.

[0098] Figure 5 The data analysis results of high affinity nucleic acid sequences generated by the present invention and RaptGen are shown. The three figures represent three different datasets. The vertical axis represents the activity value, initial represents the optimization starting point, and BO1 and BO2 represent different rounds of Bayesian optimization.

[0099] Figure 6 This shows the results using different dimensions of the latent vector space. The horizontal axis represents the number of iterations, and the vertical axis represents the loss. Dim represents the dimension, where 768 dimensions is the original dimension of the DNA nucleic acid language model output. Detailed Implementation

[0100] Through extensive and in-depth research, the inventors have proposed for the first time a de novo nucleic acid design model based on a nucleic acid language model. This model includes an encoder and a decoder. The encoder comprises a nucleic acid language model fine-tuned using SELEX data and a linear layer for dimensionality reduction, thereby generating a low-dimensional, low-noise latent space vector. The decoder comprises a linear layer for dimensionality enhancement, six self-attention layers, and a linear layer for outputting the results, thereby generating candidate sequences. By iteratively decoding and encoding, the model can continuously correct incorrect sequence fragments, thus generating high-quality, high-accuracy sequences. Furthermore, the model utilizes the E-HEBO algorithm and affinity prior data to filter the latent space vectors, thereby obtaining the latent space vector with optimal affinity. By iteratively selecting affinity sequences from candidate sequences and inputting them into the encoder as affinity prior data, the model can generate highly active nucleic acid aptamers. Based on this, the present invention is completed.

[0101] The affinity prior data is the latent space vector formed by the affinity sequence through an encoder. The affinity sequence can come from existing data, or it can be obtained by performing affinity detection on generated candidate sequences.

[0102] The computer system is equipped with at least one processor and a memory. The processor invokes a sequence of computer-executable instructions stored in the memory to implement the evaluation process defined in the claims. Although the flowchart describes the operation steps in a specific logical order, in actual execution, the steps may be processed in parallel, their order adjusted, or partially omitted in some cases. As long as such adjustments do not deviate from the core features of the technical solution described in the claims and achieve the same technical effect, they all fall within the scope of protection of this invention. This flexibility in execution order is determined by the programmable nature of computer instructions.

[0103] It should be understood that the specific methods and experimental conditions of the invention described below in varying degrees of detail are intended to provide a substantive understanding of the invention. Definitions of certain terms used in this specification are provided below. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0104] the term

[0105] As used herein, the terms “containing” or “including (comprise)” can be open-ended, semi-closed, or closed-ended. In other words, the terms also include “consistently made of” or “made of”.

[0106] As used herein, the term “and / or” refers to and covers any and all possible combinations of one or more of the related listed items.

[0107] As used in this article, the term "significant" means that, in a hypothesis test, the observed effect (such as the difference between the experimental and control groups) is unlikely to be caused solely by random error. A hypothesis test includes: the null hypothesis (H0), which assumes that the observed effect does not exist (such as no difference between the experimental and control groups); the p-value, which is the probability of observing the current or more extreme effect when H0 is true; and the significance threshold (α). The significance threshold is typically used to determine whether a hypothesis test is significant. Generally, the significance threshold is 0.05. If the p-value ≤ α, then H0 is rejected, meaning the observed effect exists, and the result is called "significant."

[0108] As used in this article, the term "Bayesian optimization" refers to a machine learning algorithm used for global optimization, particularly well-suited for finding optimal solutions to functions with high evaluation costs. It accelerates optimization efficiency by constructing a probabilistic model (typically a Gaussian process) of the objective function and updating the model based on current information to intelligently select the next evaluation point. This algorithm is commonly used in scenarios such as hyperparameter tuning of machine learning models, experimental design, and high-activity sequence generation.

[0109] As used in this paper, the terms "self-attention layer" and "self-attention mechanism" are used interchangeably. It is a method that allows a model to model global information at every position in an input sequence. In a self-attention mechanism layer, each element of the sequence not only focuses on its local context but also captures long-range dependencies by calculating its correlation with other elements. This mechanism is widely used in the encoder and decoder parts of the Transformer architecture and is particularly important in tasks such as language modeling and nucleic acid sequence modeling.

[0110] As used in this article, the term "token" refers to a mark or symbol that is the smallest unit into which a model segments text or a sequence in natural language processing or nucleic acid sequence modeling. In nucleic acid language models, a token typically represents a single nucleotide (such as A, T, C, G) or a combination thereof (such as a triplet) used to transform a biological sequence into an input form that the model can process.

[0111] As used in this article, the term "nucleic acid language model" refers to a class of machine learning models specifically designed for processing nucleic acid sequences (such as DNA and RNA). These models are similar to language models in natural language processing; by learning patterns, structures, and contextual information within a sequence, they can predict the next base of the sequence or generate nucleic acid sequences with specific functions. They have important applications in bioinformatics tasks such as gene synthesis and aptamer design.

[0112] As used in this article, the term "nucleic acid aptamer" is a molecule composed of a short single-stranded DNA or RNA sequence that can specifically bind to a target molecule (such as a protein or small molecule compound) through its unique three-dimensional structure. Due to their high specificity and affinity, nucleic acid aptamers have wide applications in disease diagnosis, drug delivery, and biosensors.

[0113] As used in this paper, the term "latent space" is a hypothetical, typically unobservable, multidimensional space containing latent features or attributes of the data. The high dimensionality of latent space vectors in the original nucleic acid language model leads to an exponential increase in the search space and inefficiency when directly applied to Bayesian optimization algorithms. Dimensionality reduction, mapping high-dimensional latent space vectors to a lower-dimensional space, effectively alleviates this problem.

[0114] SELEX

[0115] As used in this article, the term "SELEX (Systematic Evolution of Ligands by Exponential Enrichment)" is a method for screening nucleic acid aptamers, and the resulting sequences are called "SELEX sequences" or "SELEX data." Generally, SELEX involves the following workflow:

[0116] (1) Preparation of a random nucleic acid library: A random nucleic acid library is a library containing a large number of random DNA or RNA sequences. This library contains a large number of short nucleic acid sequences. It typically consists of oligonucleotides ranging from 20 to 100 base pairs in length, with random sequences in the central region and fixed primer regions at both ends for subsequent amplification. SELEX experiments can be used to screen this library for short sequences that can tightly bind to specific targets (e.g., proteins, cells, molecules). Typically, this random library contains hundreds of millions of different sequences, providing diversity for subsequent experiments.

[0117] (2) Binding the nucleic acid library to the target molecule: Next, these random nucleic acid library sequences are mixed with specific target molecules. Target molecules can be proteins, small molecule drugs, or even intact cells. The core objective of the experiment is to screen for nucleic acid sequences that can specifically bind to the target molecule. Most random sequences cannot bind effectively to the target, but a small percentage may exhibit high affinity and specificity, binding tightly to the target. These high-affinity sequences will be retained and gradually enriched in subsequent screening steps.

[0118] (3) Separation of bound and unbound sequences: In this step, it is necessary to separate the nucleic acid sequences that have successfully bound to the target from the unbound sequences. This is usually achieved through a series of washing steps. During these washing processes, sequences that have failed to bind to the target are effectively removed, while high-affinity sequences that have bound to the target are retained. The purpose of these washing steps is to ensure that only nucleic acid sequences that specifically bind to the target can proceed to the next round of screening.

[0119] (4) Amplification of bound sequences (PCR amplification): Nucleic acid sequences that successfully bind to the target are usually very rare, so these sequences need to be amplified using polymerase chain reaction (PCR) technology to significantly increase their quantity. PCR can replicate the originally trace amount of target sequence into a large number of sequences, ensuring that there are enough sequences available for further processing and analysis in subsequent screening steps.

[0120] (5) Repeated screening (iterative process): The amplified nucleic acid sequences are used again to screen for binding to the target. Through repeated binding, washing and amplification processes, after multiple rounds of iteration, each round will screen out sequences with weak binding force and retain those sequences that bind to the target most tightly.

[0121] (6) Screening for the best sequences: After multiple rounds of screening, the sequences are sorted according to the frequency of each unique sequence in the last round. Generally, sequences with a high frequency are considered to have strong affinity. Based on this screening principle, a small number of nucleic acid sequences that can bind to the target efficiently are finally obtained. These sequences are called "aptamers".

[0122] The model and its construction method of the present invention

[0123] This invention provides a method for constructing a nucleic acid de novo design model based on a nucleic acid language model, the method comprising the following steps:

[0124] (S1) Provide a nucleic acid dataset, which includes a SELEX dataset; the SELEX dataset includes affinity nucleic acid sequences and / or affinity-free nucleic acid sequences; input the nucleic acid dataset into an encoder.

[0125] As used herein, the term "affinity nucleic acid sequence" refers to a nucleic acid sequence that has affinity / can bind / has binding activity, while "affinity-free nucleic acid sequence" refers to a nucleic acid sequence that has no affinity / cannot bind / has no binding activity. Those skilled in the art are familiar with methods for detecting the affinity of nucleic acid sequences, thereby distinguishing between affinity-free and affinity-free nucleic acid sequences.

[0126] (S2) In the encoder, (S2.1) the nucleic acid sequences in the nucleic acid dataset are converted into multiple tokens using a tokenizer to obtain preprocessed nucleic acid sequences; (S2.2) the preprocessed nucleic acid sequences are converted into high-dimensional latent space vectors using a nucleic acid language model; and the high-dimensional latent space vectors are reduced to N-dimensional latent space vectors using a linear layer; where N is an integer from 7 to 9.

[0127] Preferably, the nucleic acid language model includes: a DNA language model and an RNA language model; preferably, the DNA language model includes DNABERT and Evo; the DNABERT is DNABERT-3mers; preferably, the RNA language model is RNAErnie. Preferably, N is 8.

[0128] This invention utilizes a nucleic acid language model to transform the preprocessed nucleic acid sequence into a high-dimensional latent space vector. It is understood that any nucleic acid language model capable of transforming a preprocessed nucleic acid sequence into a high-dimensional latent space vector falls within the scope of this invention.

[0129] This invention evaluates whether the training loss is similar to that of the original high-dimensional latent space vector by reducing the latent space vector to different dimensions. When the training loss is similar, it indicates that the latent space vector in that dimension can retain the key information in the original high-dimensional latent space to the greatest extent. The training loss is higher when N is 2 or 4. It is understood that when N is 7 or 9, the training loss may be similar to that when N is 8. Therefore, it can be predicted that the encoder with N of 7 or 9 may have similar performance to the encoder with N of 8, and thus falls within the scope of this invention.

[0130] (S3) The N-dimensional latent space vectors are filtered through the following steps:

[0131] (S3.1) Cluster all the N-dimensional latent space vectors obtained in step (S2) using a clustering algorithm, and select the central latent space vector of each cluster; (S3.2) Using the latent space vector of the affinity nucleic acid sequence as prior data, use a Bayesian optimization algorithm to search for latent space vectors with better affinity than the prior data within a preset range; and / or (S3.3) Using the latent space vector of the affinity nucleic acid sequence as prior data, select the latent space vector that is close to the latent space vector of the affinity nucleic acid sequence in distance; input the latent space vectors generated in steps (S3.1), (S3.2) and / or (S3.3) into the decoder.

[0132] Those skilled in the art are familiar with various clustering methods, Bayesian optimization algorithms, and distance algorithms. They are also familiar with how to select the latent space vector of the cluster center based on the clustering results, or how to select the latent space vector based on a preset range or a certain distance. Generally, common clustering methods include Hidden Markov Models (HMMs), soft clustering (such as fuzzy C-means, Gaussian mixture clustering), etc. Preferably, the clustering method is Gaussian mixture clustering. Common Bayesian optimization algorithms include HEBO. This invention develops E-HEBO based on HEBO. This algorithm uses only the optimal point among known function values ​​in each Gaussian process modeling, that is, the sequence with the best binding performance and its affinity among all known active nucleic acid sequences. At the same time, when defining the search space, it sets a progressively shrinking search radius centered on the embedding corresponding to the optimal function value to continuously improve the affinity. Preferably, the Bayesian optimization algorithm is E-HEBO. Common distance algorithms include Euclidean distance, Manhattan distance, cosine distance, etc. Preferably, the distance algorithm is Euclidean distance.

[0133] As used herein, the term "preset range" is a threshold or region range set based on the affinity of the prior data. In this invention, when the affinity of the prior data is high, the range is ±10 for each dimension of the latent space vector; when the affinity of the prior data is low, the range is ±40 for each dimension of the latent space vector. It is understood that the range set based on the affinity of the prior data can be changed according to actual circumstances. For example, when the affinity of the prior data is high, the range can be ±1, ±2, ±3, ±4, ±5, ±6, ±7, ±8, ±9, ±10 for each dimension of the latent space vector; when the affinity of the prior data is low, the range can be ±1, ±2, ±3, ±4, ±5, ±6, ±7, ±8, ±9, ±10, ±20, ±30, ±40 for each dimension of the latent space vector. Those skilled in the art are familiar with methods for detecting the affinity of nucleic acid sequences, and thus know how to distinguish between nucleic acid sequences with high affinity and those with low affinity based on the nucleic acid sequence affinity results. In this invention, the term "low affinity" refers to a binding specificity score <10, such as <5 or <0; the term "high affinity" refers to a binding specificity score >10, such as >15 or >20.

[0134] As used herein, the term "close in distance" refers to a range within a certain distance threshold, which is a suitable, relatively small range; this range can be changed according to actual circumstances. The number of latent space vectors that are close in distance to the latent space vector of the affinity nucleic acid sequence can be ≥1, ≥10, ≥20, ≥30, ≥40, ≥50, ≥60, ≥70, ≥80, ≥90, ≥100, ≥150, such as 100.

[0135] Steps (S3.1), (S3.2), and (S3.3) can be combined, and can be performed simultaneously or in any order. Finally, the selected latent space vector is input into the decoder.

[0136] (S4) In the decoder, the latent space vector is mapped to a probability distribution corresponding to each symbol using a network model, in order to generate candidate sequences and the predicted probabilities of each symbol.

[0137] Those skilled in the art are familiar with various network models. Generally, common network models include self-attention layers, multilayer perceptrons (MLPs), linear layers, convolutional neural networks (CNNs), recurrent neural networks (RNNs), Mamba, Hyena, and GANs. Preferably, the network model is a self-attention layer. Preferably, the decoder specifically includes the following steps:

[0138] (S4.1) The latent space vector is increased in dimension using a linear layer to obtain an increased-dimensional latent space vector; the increased-dimensional latent space vector is then input into a multi-head self-attention layer.

[0139] (S4.2) Perform the following steps in each subspace of the multi-head self-attention layer:

[0140] (S4.2a) Transform the increased-dimensional latent space vector into a query matrix, a key matrix, and a value matrix;

[0141] (S4.2b) Calculate the similarity score matrix based on the query matrix and the key matrix;

[0142] (S4.2c) Normalize the similarity score matrix using an activation function to obtain the attention weight matrix;

[0143] (S4.2d) Multiply the attention weight matrix and the value matrix to obtain the self-attention output of each subspace;

[0144] (S4.3) The self-attention outputs of each subspace are concatenated and mapped through a linear layer to generate a probability distribution corresponding to each symbol.

[0145] Furthermore, the method also includes iterative encoding and decoding, as well as iterative generation of affinity prior data to obtain high-quality, high-accuracy, and high-activity candidate sequences, which ultimately constitute the nucleic acid de novo design model based on the nucleic acid language model.

[0146] (S5) The nucleic acid de novo design model based on the nucleic acid language model is formed through iterative optimization via the following steps: (S5.1) Sequence optimization iteration: Multiple probability distribution-based sampling is performed on each symbol output in step (S4) to obtain multiple candidate sequences; the multiple candidate sequences are scored using the nucleic acid language model, and the sequence with the highest score is selected as the final sequence. As used herein, the term "probability distribution-based sampling" refers to the process of randomly generating sample points (data) from a specified probability distribution (such as a normal distribution, a uniform distribution, or a complex custom distribution) according to its probability rules.

[0147] And / or perform (S5.2) prior data iteration: (S5.2a) detect the affinity of the candidate sequences and divide the candidate sequences into affinity candidate sequences and non-affinity candidate sequences; (S5.2b) use the result of step (S5.2a) as the nucleic acid dataset, iterate steps (S2)-(S4) and (S5.2a) multiple times, and during iteration, add the latent space vector of the affinity candidate sequences to the prior data in step (S3.2) and / or (S3.3) to obtain the final sequence.

[0148] The nucleic acid de novo design model based on a nucleic acid language model of the present invention includes an encoder, a screening, a decoder, and an iterative optimization step. Using the nucleic acid de novo design model of the present invention, high-quality, high-accuracy, and high-activity nucleic acid sequences can be obtained.

[0149] The main advantages of this invention include:

[0150] (1) This invention combines the advantages of two existing models, RaptRanker and RaptGen, which can both find low-frequency high-binding sequences in nucleic acid libraries and generate SELEX sequences.

[0151] (2) This invention uses a pre-trained nucleic acid language model as a basis for fine-tuning, which can create a low-noise hidden space. Compared with RaptGen, the model of this invention is more robust and less susceptible to data noise.

[0152] (3) In this invention, the feature dimension of a nucleic acid sequence of length L is eight-dimensional (8L), while the feature dimension of a sequence in RaptGen is only two-dimensional. Therefore, this invention can fully capture the contextual semantic information of the nucleic acid sequence.

[0153] (4) Compared with the ordinary Bayesian optimization of RaptGen, the E-HEBO algorithm used for the first time in this invention can alleviate heteroscedasticity and nonstationarity in the high-dimensional optimization space, thereby generating high-affinity nucleic acid sequences more efficiently.

[0154] The present invention will be further illustrated below with reference to specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. It is also understood that the purpose of describing the present invention in conjunction with the embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a deep understanding of the invention, many specific details will be included in the following description. The invention may also be practiced without using these details. Furthermore, to avoid confusion or obscuring the focus of the invention, some specific details will be omitted in the description.

[0155] Example 1: Construction of the Model of the Invention

[0156] This invention employs an encoder-decoder architecture. Figure 1 In this model, the encoder processes the input data and extracts features, while the decoder generates the output based on these features. In the model of this invention, the encoder embeds the proposed E-HEBO algorithm to achieve live-guided conditional generation.

[0157] (1) Encoder:

[0158] Unlike RaptGen, which uses a VAE network trained from scratch, the encoder of this invention uses a nucleic acid language model fine-tuned with SELEX data: DNABERT is selected when processing DNA sequences, while RNAErnie is selected when processing RNA sequences.

[0159] DNABERT is a model specifically designed for processing DNA sequence data. It gains a general understanding of DNA semantic information by pre-training on the human genome through mask prediction. DNABERT can be divided into 3mers, 4mers, 5mers and 6mers versions according to different word segmentation rules. Here we choose DNABERT-3mers.

[0160] RNAErnie was pre-trained on a large number of RNA sequences for mask prediction, thereby mastering a general understanding of RNA semantic information.

[0161] This general DNA or RNA semantic knowledge can penalize the model for overfitting to the SELEX data when training on the SELEX data, thereby creating a low-biased hidden space that mitigates the noise problem present in the SELEX sequencing data.

[0162] Furthermore, to alleviate the inefficiency of the optimization algorithm in high-dimensional search spaces and thus reduce the optimization difficulty of the E-HEBO algorithm in the hidden space, additional linear layers are added to reduce dimensionality. In this invention, these linear layers compress the encoder's output dimension to 8 dimensions per token. Here, a token can be understood as a basic unit in a sequence. For example, in DNABERT, each token is composed of the kmers bases of the DNA sequence, such as ACT, ATG, etc., while in RNAErinie, each token is a single RNA base A, U, C, G. In addition to these sequence tokens, there are also some tokens with special functions, such as the sequence start token—[CLS], the sequence end token—[EOS], and the sequence padding token—[PAD].

[0163] The entire process of the SELEX data input / output encoder is as follows:

[0164] input = Tokenizer RNA / DNA (Nucleic acid sequence)

[0165]

[0166] DNA or RNA sequences are converted into computer-processable numbers using their corresponding tokenizers. The processed sequences are then input into a fine-tuned nucleic acid language model (DNABERT or RNAErinie) to obtain the unreduced latent space vectors. These vectors are then passed through a linear dimensionality reduction layer to reduce the dimension of each token to 8 dimensions. That is, the final latent space vector.

[0167] (2) Decoder:

[0168] The decoder consists of six self-attention layers. The self-attention mechanism allows the model to dynamically focus on information at different positions within the sequence, making the generated content more context-sensitive. The latent space vectors are used by the decoder to generate new sequences. The decoder's process for generating new sequences is as follows:

[0169]

[0170] Generate sequence = Linear(h) las tlayer ).

[0171] The latent space vector is obtained by upscaling the 8-dimensionality of each token to D-dimensionality through a linear layer. Then, it enters a 6-layer self-attention layer to obtain the final hidden vector h. las tlayer Finally, predictions are generated by mapping to a dictionary through a linear layer.

[0172] The self-attention layer employs a multi-head self-attention mechanism. Its calculation process is as follows:

[0173] (1) Input latent space vector (where T is the sequence length, d) model The vector dimension is input into three independent linear mapping layers to obtain the query matrix Q, the key matrix K, and the value matrix V, which are calculated using the following formula:

[0174] Q = XW Q +b Q K = XW K +b K V = XW V +b V ,

[0175] Among them W Q W K W VFor the weight matrix b Q ,b K ,b V This is the bias vector.

[0176] (2) Calculate the similarity score matrix between the query and the key:

[0177]

[0178] Where d k Let be the dimension of the key vector. As a scaling factor.

[0179] (3) Perform Softmax normalization on the score matrix row by row to obtain the attention weight matrix A:

[0180] A = Softmax(Scores),

[0181] (4) Multiply the attention weight matrix by the value matrix to obtain the self-attention output:

[0182] Attention(Q,K,V) = AV

[0183] (5) In the multi-head mechanism, the input is split into multiple subspaces and the above process is executed in parallel. Then the outputs of each head are concatenated and the final output is obtained through a linear mapping layer.

[0184] The fully connected linear layer is used to implement dimensionality transformations of latent space vectors, including dimensionality increase and decrease. Its calculations are all performed using affine transformations, with the specific formula as follows:

[0185] Y = XW + b,

[0186] in, For the input vector, This is the weight matrix. As the bias term, d in With d out These are the dimensions of the input and output, respectively.

[0187] (3)E-HEBO (Heteroscedastic and Evolutionary Bayesian Optimisation) algorithm:

[0188] The HEBO algorithm is a variant of the Bayesian algorithm that mitigates the heteroscedasticity and non-stationarity of the black-box function between the latent space vector and the affinity of nucleic acid sequences. However, HEBO is prone to getting trapped in local optima in high-dimensional embeddings, leading to reduced optimization performance. Therefore, this invention innovatively proposes E-HEBO (Evolutionary Heteroscedastic and Evolutionary Bayesian Optimisation) in the latent space of the model to further improve performance. E-HEBO uses only the optimal point among known function values ​​in each Gaussian process modeling, namely the sequence with the best binding performance and its affinity among all known active nucleic acid sequences. Simultaneously, when defining the search space, it sets a progressively shrinking search radius centered on the embedding corresponding to the optimal function value to continuously improve affinity.

[0189] The process of using E-HEBO to find the latent vector with optimal affinity is as follows:

[0190] First, select a high-affinity nucleic acid sequence of length L from the nucleic acid sequences with affinity data. Encode this nucleic acid sequence using the encoder of this invention to obtain its latent space vector Z, with a dimension of 8L. Input the Z vector and the sequence affinity into the E-HEBO algorithm and define the search range. When the affinity is good, the range is set to ±10 for each dimension of the Z vector; when the affinity is low, the range is set to ±40 for each dimension of the Z vector. Other parameters are set to the default parameters of the HEBO algorithm for the search, and finally, the next latent space vector Z that may result in the highest affinity is found. next .

[0191] S1: Model Training. This invention provides a two-stage training method specifically designed for sequence generation based on nucleic acid language models. The entire training process is divided into two stages, aiming to fully utilize SELEX experimental data and improve the model's performance in SELEX generation tasks.

[0192] Phase 1: In the first phase of training, this invention utilizes a nucleic acid model (such as DNABERT or RNAErnie) for continuous pre-training. Specifically, a Masked Language Model (MLM) approach is employed, randomly masking the input nucleic acid sequence at a ratio of 15%, meaning 15% of the bases in the sequence are randomly masked. The model then continuously optimizes its parameters by predicting the bases at these masked positions. During this process, the SELEX (Systematic Evolution of Ligands by Exponential Enrichment) dataset is used as training data to ensure the model can capture the sequence features and contextual information from the SELEX data.

[0193] Phase Two: In the second phase of training, this invention proposes an innovative strategy that employs a sequence reconstruction task to further endow the model with generative capabilities. In this phase, the encoder portion of the model remains frozen to retain the knowledge gained in the first phase. The decoder portion, however, is randomly initialized to learn from scratch how to reconstruct nucleic acid sequences from the latent representations output by the encoder. The decoder training is still based on SELEX data, ensuring it learns features closely related to sequence structure and optimizes for the nucleic acid sequence generation task by reconstructing complete sequences. By freezing the encoder and focusing on the decoder training, this process ensures that the model, in addition to its original semantic understanding of nucleic acid sequences, is endowed with the ability to generate sequences.

[0194] S2: Model Inference. The inference process of this invention first obtains a meaningful hidden space vector through the encoder, such as the cluster center vector obtained by clustering or the vector of the next likely high-activity sequence obtained by E-HEBO. Subsequently, the decoder decodes the hidden vector into the predicted sequence seq. first The system then performs feedback optimization based on the low-probability parts of the generated sequence to obtain the final sequence seq. final The specific process is as follows:

[0195] The latent space vector Z is fed into the decoder of this invention to obtain the first generated sequence seq. first and the probability P of each of its tokens first For the first generated seq firstThere may be special token characters such as [CLS], [EOS], [PAD], etc. These special tokens at the positions where bases should appear are masked. The first and last tokens of the sequence are forcibly set to [CLS] and [EOS] tokens, respectively. The processed sequence is fed into the encoder of this invention for mask position prediction. During prediction, the probability distribution of special tokens is masked, and only the distribution of nucleic acid bases is predicted. After prediction, the nucleic acid token at the mask position and its predicted probability are obtained. The nucleic acid token and predicted probability are then used to replace the seq at the corresponding positions. first and P first The optimized sequence seq was obtained. second and P second Then, an optimization mask number is set, which determines the number of symbols that need to be re-predicted and updated in each iteration. Typically, symbols are sorted from lowest to highest prediction probability, and the symbols with the lowest probability are selected first for masking to focus on optimizing the most uncertain parts of the sequence. This mask number is generally set to 3, based on P. second The lowest probability symbols are selected and masked, then fed into the encoder for iterative optimization. The number of iterations is typically set to 3, ultimately yielding the generated sequence seq. final .

[0196] Through this iterative optimization mechanism, the model continuously corrects uncertain sequence fragments until high-quality, high-accuracy sequences are generated. This process ensures that the generated sequences possess high reliability and biological function.

[0197] The steps for generating highly active nucleic acid aptamers in the SELEX experiment according to this invention are as follows:

[0198] S1: Use the last round of SELEX sequencing data to fine-tune DNABERT or RNAErnie. The proxy task uses a random mask with a mask probability of 15% to obtain the fine-tuned DNABERT or RNAErnie, which serves as the encoder of this invention.

[0199] S2: Freeze the encoded weight parameters trained in a, and add a self-attention layer as a decoder. Use the same training data in a to perform full sequence reconstruction to train the decoder.

[0200] S3: Input all SELEX sequence data into the encoder to obtain the latent space vectors of all sequences, and perform Gaussian mixture clustering on these vectors to divide them into 10 classes. For each class, take the vector of its cluster center as the input of the decoder for decoding, and then send it to the encoder for iterative optimization to obtain 10 sequences.

[0201] S4: If a sequence with Kd values ​​exists, these sequences can be fed into the encoder of this invention to obtain their latent space vector Z. center Then, all SELEX data sequences are fed into the encoder of this invention to obtain all latent space vectors Z. all Find Z based on Euclidean distance all Mid-range Z center The 100 most recent latent space vectors are used to identify their corresponding nucleic acid sequences as candidate sequences. These 100 sequences are then ranked according to the Gibbs energy of the secondary structure predicted by Mfold (or other screening methods are used), and the sequence with the lowest Gibbs energy is selected as the final sequence. This process allows the present invention to further screen low-frequency, high-binding sequences from the library.

[0202] S5: Perform Kd affinity tests on the top 10 most frequent sequences in the SELEX sequence data and the sequences obtained in S3 and S4 to obtain the Kd values ​​of all sequences. Some of these sequences may not have Kd values ​​and are considered to have no affinity.

[0203] S6: Use sequences with known Kd values ​​as prior knowledge. For each sequence with a Kd value, use E-HEBO to obtain 100 latent space vectors that may generate highly active sequences. Use a decoder to decode these vectors and use an encoder to iteratively optimize them to generate 100 new sequences. Use the nucleic acid secondary structure prediction software Mfold to predict the secondary structure of these 100 sequences, and select the sequence with the lowest Gibbs free energy as the Kd sequence to be tested.

[0204] S7: Measure the Kd value of the new sequence generated in S6, and iterate through S6 and Kd measurement for optimization.

[0205] Example 2: Comparison of the semantic representation capabilities of the encoders of this invention and RaptGen on different datasets.

[0206] The semantic richness of the encoder of this invention and the encoder of RaptGen were evaluated using the public SELEX dataset of AJD. The evaluation process is as follows:

[0207] (1) RaptGen and the encoder of the present invention are trained using SELEX data. The encoder of the present invention uses full fine-tuning to fine-tune the nucleic acid language model. Since the public dataset is DNA SELEX data, DNABERT-3mers is used for fine-tuning.

[0208] (2) 200 sequences were randomly sampled from the SELEX data, and each pair of sequences was combined to form a total of 19,900 sequences. Given a sequence pair, calculate the edit distance for all sequence pairs.

[0209] (3) Input all the sequences from (2) into the RaptGen encoder trained in (1) and the finely tuned DNABERT-3mers of this invention, freeze the network weight parameters, and obtain the latent space representation of each sequence in RaptGen and this invention. Calculate the 19900 in (2) based on these latent space representations. The cosine similarity of each sequence pair was used to obtain two sets of cosine similarities, which were calculated by RaptGen and the encoder of this invention, respectively.

[0210] (4) Calculate two sets of Pearson correlation coefficients based on the two sets of cosine similarities obtained in (3) and the edit distance in (2), which represent the correlation between RaptGen and the latent space representation and edit distance of this invention, respectively. The greater the correlation, the richer the semantic information of its representation.

[0211] The results are as follows Figure 2 As shown, the semantic representation results of this invention and RaptGen on multiple SELEX public datasets are compared. It is clear that this invention demonstrates superior semantic representation capabilities on all datasets. This advantage is attributed to the use of a nucleic acid language model as the encoder, which makes it far superior to RaptGen in capturing and representing the semantic features of nucleic acid sequences, exhibiting stronger robustness and accuracy.

[0212] Example 3: Comparison of the sequence generation capabilities of the present invention and RaptGen's decoder on different datasets.

[0213] To evaluate the decoding capability of the decoder of this invention, experiments were conducted to assess its decoding performance compared to RaptGen. The experimental procedure is as follows:

[0214] (1) Train RaptGen and the model of this invention using SELEX data, and input all the SELEX data into the two models to obtain their latent space vectors respectively.

[0215] (2) Kernel density estimation is used to sample the latent space vectors of RaptGen and the present invention. Since RaptGen generates sequences slowly, the number of samples is set to 2000, and 2000 latent space vectors are obtained for RaptGen and the model of the present invention respectively.

[0216] (3) Use RaptGen and the decoder of this invention to decode the above 2000 latent space vectors respectively, thereby generating 2000 sequences.

[0217] (4) Randomly sample 2000 sequences from the SELEX data.

[0218] (5) Calculate the frequencies of 3, 4, 5 and 6 mers in the three sets of sequences: RaptGen, the present invention and random sampling. Then, calculate the correlation between the frequencies of 3-6 mers generated by RaptGen and the present invention and the frequencies of the random sampling sequences to obtain the Pearson correlation coefficient.

[0219] The results of the above experiments performed on different SELEX datasets are as follows: Figure 3 The present invention can obtain higher KMRes feature correlation with random sample sequences, which indicates that the present invention can decode and reconstruct sequences better than RaptGen.

[0220] Example 4: Comparison of results using E-HEBO or HEBO algorithms

[0221] This invention experimentally verifies the search efficiency of the HEBO and E-HEBO algorithms in the hidden space of this invention. On the GATA4 dataset, this invention performs 10 rounds of activity-guided sequence generation, generating 10 sequences in each round, and compares sequences using different activity sequences as prior knowledge starting points. Figure 4 It is evident that E-HEBO can yield sequences with better activity values ​​and faster convergence speed under conditions of high or medium activity prior knowledge. In practical applications of nucleic acid aptamer generation, medium or high activity sequences are generally used as the starting point for optimization. Therefore, E-HEBO is used for the optimization and generation of nucleic acid aptamers.

[0222] Example 5: Comparison of high-affinity nucleic acid sequence data analysis results generated by the present invention and RaptGen

[0223] This invention was validated through data analysis on three different SELEX datasets.

[0224] Compared to RaptGen, this invention can find a better starting optimization point. This is attributed to the fact that this invention not only uses cluster centers to generate sequences, but also uses sequences with known activity values, such as the top 10 high-affinity sequences, thereby utilizing the low-frequency high-affinity sequences selected from the top 10 sequences. In subsequent optimization stages, this invention can generate higher-activity sequences on any dataset. Figure 5 This demonstrates the powerful application of the present invention in the generation of nucleic acid aptamers.

[0225] Example 6: Selection of the dimension of the latent vector space

[0226] In this invention, the output of nucleic acid language models typically has high dimensionality; for example, the DNABERT model has an output dimension of 768. Directly using such high-dimensional data for Bayesian optimization leads to an excessively high dimensional search space, significantly reducing optimization efficiency and effectiveness. Therefore, it is necessary to appropriately compress the dimensionality of the nucleic acid language model's output to balance computational efficiency and information preservation.

[0227] Experiments were conducted to compare the training loss performance when the latent space dimension for each token was set to 2, 4, 8, and 768 during pre-training on the GATA4 dataset. Figure 6 ).

[0228] The results show that when the dimension is set to 8, the model's training loss can reach a level similar to that of the original 768-dimensional model, while it is difficult to achieve a low training loss when the dimensions are 2 and 4. This result demonstrates that compressing the dimension to 8 not only significantly reduces the search space complexity of Bayesian optimization but also preserves the key information in the original high-dimensional latent space to the greatest extent, thereby effectively improving the performance and stability of the optimization process.

[0229] All documents mentioned in this invention are incorporated herein by reference as if each document were individually incorporated by reference. Furthermore, it should be understood that after reading the foregoing teachings of this invention, those skilled in the art can make various alterations or modifications to this invention, and these equivalent forms also fall within the scope defined by the appended claims.

Claims

1. A method for constructing a de novo nucleic acid design model based on a nucleic acid language model, characterized in that, The method includes the following steps: (S1) Provide a nucleic acid dataset, the nucleic acid dataset including a SELEX dataset; the SELEX dataset includes affinity nucleic acid sequences and / or affinity-free nucleic acid sequences; input the nucleic acid dataset into an encoder; (S2) In the encoder, (S2.1) Use a tokenizer to convert the nucleic acid sequences in the nucleic acid dataset into multiple tokens to obtain preprocessed nucleic acid sequences; (S2.2) The preprocessed nucleic acid sequence is converted into a high-dimensional latent space vector using a nucleic acid language model; the high-dimensional latent space vector is reduced to an N-dimensional latent space vector using a linear layer, where N is an integer from 7 to 9; (S3) The N-dimensional latent space vectors are filtered through the following steps: (S3.1) Use a clustering algorithm to cluster all the N-dimensional latent space vectors obtained in step (S2), and select the central latent space vector of each cluster; (S3.2) Using the latent space vector of the affinity nucleic acid sequence as prior data, a Bayesian optimization algorithm is used to search for latent space vectors with higher affinity than the prior data within a preset range; and / or (S3.3) Using the latent space vector of the affinity nucleic acid sequence as prior data, select a latent space vector that is close to the latent space vector of the affinity nucleic acid sequence in distance; Input the latent space vectors generated in steps (S3.1), (S3.2) and / or (S3.3) into the decoder; (S4) In the decoder, the latent space vector is mapped to a probability distribution corresponding to each symbol using a network model, in order to generate candidate sequences and the predicted probabilities of each symbol. (S5) The nucleic acid de novo design model based on the nucleic acid language model is formed by iterative optimization through the following steps: (S5.1) Sequence optimization iteration: Multiple probability distribution-based sampling is performed on each symbol output from step (S4) to obtain multiple candidate sequences; the nucleic acid language model is used to score the multiple candidate sequences, and the sequence with the highest score is selected as the final sequence; and / or (S5.2) Prior data iteration: (S5.2a) Detect the affinity of the candidate sequences and divide the candidate sequences into affinity candidate sequences and non-affinity candidate sequences; (S5.2b) Using the result of step (S5.2a) as the nucleic acid dataset, iterate steps (S2)-(S4) and (S5.2a) multiple times. During iteration, add the latent space vector of the affinity candidate sequence to the prior data in step (S3.2) and / or (S3.3) to obtain the final sequence. The nucleic acid language model includes: a DNA language model and an RNA language model; the DNA language model is DNABERT, and the RNA language model is RNAErnie.

2. The method as described in claim 1, characterized in that, The DNABERT is selected from the group consisting of DNABERT-3mers, DNABERT-4mers, DNABERT-5mers, or DNABERT-6mers; and N=8.

3. The method as described in claim 1 or 2, characterized in that, The Bayesian optimization algorithm is selected from the following group: E-HEBO algorithm, HEBO algorithm, or other Bayesian optimization-based algorithms; wherein, the E-HEBO algorithm uses only the sequence with the best binding performance and its affinity among all known affinity nucleic acid sequences as the optimal function value in each Gaussian process modeling, and at the same time, when defining the search space, it sets a progressively smaller search radius centered on the embedding corresponding to the optimal function value to continuously improve the affinity.

4. The method according to any one of claims 1, characterized in that, The network model includes self-attention layers, multilayer perceptrons (MLP), linear layers and convolutional neural networks (CNN), recurrent neural networks (RNN), Mamba, Hyena, GAN, or combinations thereof.

5. A method for generating highly active nucleic acid aptamers, characterized in that, The method includes the following steps: (X1) Provide a nucleic acid dataset, which includes a SELEX dataset; the SELEX dataset includes affinity nucleic acid sequences and / or affinity-free nucleic acid sequences; input the nucleic acid dataset into an encoder; (X2) In the encoder, a tokenizer is used to convert the nucleic acid sequences in the nucleic acid dataset into multiple tokens to obtain preprocessed nucleic acid sequences; the preprocessed nucleic acid sequences are converted into high-dimensional latent space vectors using a continuously pre-trained nucleic acid language model; the high-dimensional latent space vectors are reduced to N-dimensional latent space vectors using a linear layer, where N is an integer from 7 to 9; the nucleic acid language model includes: a DNA language model and an RNA language model; the DNA language model is DNABERT, and the RNA language model is RNAErnie; (X3) Cluster all the N-dimensional latent space vectors obtained in step (X2) using a clustering algorithm, select the center latent space vector of each cluster and input it into the decoder to generate candidate sequences; input the candidate sequences into the encoder for iterative optimization to obtain optimized sequences; (X4) Using the latent space vector of a nucleic acid sequence with known affinity as prior data, the latent space vector closest to the prior data is determined based on spatial distance and input into the decoder to determine the corresponding nucleic acid sequence; the nucleic acid sequence is screened based on its properties, and the nucleic acid sequence whose properties reach a preset level is selected as the preferred sequence; (X5) Detect the affinity of the top 10-100 nucleic acid sequences in the SELEX dataset, the optimized sequences obtained in step (X3), and the preferred sequences obtained in step (X4), and classify them into affinity sequences and non-affinity sequences; (X6) Input the nucleic acid sequences obtained in (X5) into the encoder to obtain the latent space vector of each sequence; using the nucleic acid sequences with known affinity and the latent space vectors of the affinity sequences as prior data, use the Bayesian algorithm to search for latent space vectors that are close to the prior data within a preset range, and input them into the decoder to generate candidate sequences; input the candidate sequences into the encoder for iterative optimization to obtain optimized sequences; screen according to the properties of the optimized sequences, and select the optimized sequences whose properties reach a preset level as the test sequences; (X7) Detect the affinity of the test sequence and divide the test sequence into affinity test sequences and non-affinity test sequences; (X8) Iterate through steps (X6)-(X7) multiple times. During iteration, the latent space vector of the affinity test sequence is added to the prior data in step (X6) to obtain a highly active nucleic acid aptamer.

6. The method as described in claim 5, characterized in that, Before (X1), the following step (X0) is included: Training the nucleic acid language model and decoder in the encoder, including the following steps: (X0.1) Provide SELEX data for model training, wherein the SELEX data is the last round of SELEX sequencing data; (X0.2) The SELEX data is masked with a preset masking rate, and the nucleic acid language model is continuously pre-trained on the masked SELEX data to obtain a continuously pre-trained nucleic acid language model; wherein the preset masking rate is 10%-20%; (X0.3) Keeping the training result obtained in step (X0.2) unchanged, the decoder is trained on the SELEX data to obtain the trained decoder.

7. A de novo nucleic acid design system based on a nucleic acid language model, characterized in that, The system includes: An input unit is configured to input data, the data being a SELEX dataset, the SELEX dataset comprising affinity nucleic acid sequences and / or affinityless nucleic acid sequences; The design unit is configured as a nucleic acid de novo design model based on a nucleic acid language model, which generates highly active nucleic acid aptamers based on the SELEX dataset, thereby outputting the results of the design unit; wherein the model is constructed by the method of claim 1. An output unit is configured to output the results of the design unit.

8. An electronic device comprising a processor and a memory, characterized in that, The memory contains multiple executable instructions, and the processor is used to read the instructions and execute the steps in the method as described in claim 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when read and executed by a processor, implement the steps of the method as described in claim 5.

10. A computer program product comprising computer-executable instructions, characterized in that, When the computer-executable instructions are executed by the processor, they implement the steps of the method of claim 5.

Images