Function annotation abundance sequence-based base model training method and device

Abstract

The invention relates to a base model training method and device based on a functional annotation abundance sequence. The method comprises the following steps: S1, carrying out function annotation on an open reading frame of a genome or metagenome sample; s2, counting the occurrence frequency of each annotation and constructing a sequence according to an abundance descending order; s3, after the sequence is subjected to token processing, inputting the sequence into a model based on Transform, and adopting joint training of language modeling, comparative learning and classification loss to obtain species-level and token-level fixed dimension embedding; s4, on the basis of the embedding, completing downstream tasks such as phylogenetic tree construction, species identification and phenotype prediction, BGC / MGC recognition and key gene positioning in three levels, namely a genome level, a gene cluster level and a gene / protein level. In the embodiment of the invention, good uniformity, expandability and interpretability are displayed, and the dependence on a reference database is reduced. The corresponding device comprises a data processing module, a model training module and an application module, and can be realized by program instructions in a computer readable storage medium.

Description

A method and apparatus for training a pedestal model based on functional annotation abundance sequences Technical Field

[0001] This invention relates to the interdisciplinary field of bioinformatics and artificial intelligence, specifically to a method and apparatus for training a pedestal model based on functional annotation abundance sequences, and its applications in phylogenetic tree construction, species identification and phenotypic prediction, identification of functional gene clusters (including BGC / MGC) and localization of key genes. Background Technology

[0002] As the most widespread and abundant life form on Earth, microorganisms are the core drivers of material cycling and energy flow in ecosystems. They participate in the global cycling and transformation of elements such as carbon, nitrogen, and phosphorus, maintaining ecological functions such as soil fertility and water purification. Simultaneously, microorganisms are closely related to human health. The human gut microbiota influences host health by regulating processes such as nutrient absorption and immune response, while the spread of pathogenic microorganisms is a major cause of infectious disease outbreaks. Furthermore, microorganisms have irreplaceable value in the industrial field, playing a crucial role in antibiotic synthesis, biofuel production, and the degradation of environmental pollutants. Therefore, phylogenetic analysis, species identification, and functional gene cluster identification of microorganisms are the core foundation for a deeper understanding of microbial ecological functions, safeguarding human health, and promoting the industrial application of microbial resources. The accuracy and efficiency of this analysis directly determine the depth and application value of related research. Since the beginning of the 21st century, the rapid iteration and widespread adoption of next-generation sequencing (NGS) technology have fundamentally changed the research paradigm in microbiology and genomics. From early single-gene fragment analysis based on Sanger sequencing to the large-scale application of technologies such as Illumina, PacBio, and Nanopore, sequencing throughput has leaped from "kilobase pairs (kbp)" to "taptobase pairs (Tbp)," while sequencing costs have decreased by more than 100,000 times. This technological revolution has led to an explosive growth in microbial genome data: according to statistics from the NCBI GenBank database, as of 2024, more than 1 million microbial genome sequences have been collected, and metagenomic sequencing data (reflecting the overall genomic information of microbial communities in the environment) is growing at a rate of PB per year (1PB=1024TB), with increasingly rich data types, including not only genome sequences but also transcriptome (gene expression information), metabolome (metabolic product information), proteome (protein expression information), and other multi-omics data. This massive amount of multidimensional data makes it possible to comprehensively analyze the structure and functional characteristics of microbial communities, but it also poses serious challenges to data processing and analysis methods.Traditional microbial analysis methods were largely developed during the era of low-throughput sequencing, and their design concepts and technical frameworks are ill-suited to the current demands for processing massive amounts of multi-omics data. On the one hand, traditional methods generally suffer from low data processing efficiency—for example, phylogenetic analysis methods based on single-gene sequences, when faced with datasets containing tens of thousands of genome sequences, require several days or even weeks just for sequence alignment, far from meeting the needs of rapid research. On the other hand, traditional methods exhibit significant shortcomings in core capabilities such as multi-dimensional information integration, unknown species discovery, and functional gene analysis. At the information integration level, traditional methods are often limited to single-type data (such as using only genome sequences or only functional annotation information), failing to effectively integrate multi-dimensional data such as the evolutionary characteristics of genome sequences, the biological significance of functional annotations, and gene abundance expression regulation information, leading to biased analysis results. At the level of unknown species discovery, traditional culture techniques can only isolate and culture about 1% of microorganisms in nature, while traditional identification methods (such as 16S rRNA)... Sequencing relies on reference sequences of known species, making it difficult to identify new species that have not been cultured and lack reference information from metagenomic data. At the functional gene analysis level, traditional methods are mostly based on the sequence similarity of known functional genes for annotation. For genes or gene clusters with novel structures and unknown functions (such as novel metabolic gene clusters), they often cannot accurately analyze their functions, resulting in the omission of a large number of potential microbial functional resources.

[0003] In microbiology and genomics research, phylogenetic analysis, species identification, and functional gene cluster identification are core tasks. Existing methods mainly have the following shortcomings: 1. Phylogenetic tree construction: relies on alignment and substitution models of single-copy orthologous genes; it is difficult to integrate information on multi-copy genes, paralogous genes, gene family expansion, and functional context.

[0004] 2. Species identification: 16S rRNA or genome-wide average nucleotide identity (ANI) methods are commonly used; however, they rely heavily on reference databases and have limited ability to identify new species.

[0005] 3. Gene cluster identification: Tools such as ClusterFinder rely on templates, and deep learning methods such as DeepBGC mainly identify BGC (biosynthetic gene clusters) but cannot cover MGC (metabolic gene clusters); existing methods are mostly "black boxes" and lack interpretable identification capabilities for key genes.

[0006] Meanwhile, although large-scale language foundation models have made progress in natural language and protein sequence analysis, there is still no integrated model for functional annotation abundance sequences, and there is also a lack of systematic parameter optimization and scaling law verification. Summary of the Invention

[0007] This invention aims to address the limitations of existing technologies in phylogenetic tree construction, species identification, and functional gene cluster recognition, and provides a pedestal model training method and apparatus based on functional annotation abundance sequences, which takes into account uniformity, generalization, scalability, and interpretability.

[0008] The objective of this invention is achieved through the following technical solution: a base model training method based on functional annotation abundance sequences, comprising: S1: performing bioinformatics functional annotation on protein-coding sequences of a species' genome or metagenomic sample; the functional annotation includes one or more of OG, GO, KO, and Pfam, wherein OG represents a group of orthologous genes derived from a common ancestor gene, reflecting gene family expansion / contraction; GO is an ontology label reflecting gene function and pathway; KO is used to reveal metabolic networks and signal regulation; Pfam represents a family label at the protein domain level; S2: counting the occurrence frequency of the functional annotations in the sample and arranging them in descending order of abundance to construct the functional annotation abundance sequence of the sample; S3: tokenizing the abundance sequence and inputting it into a Transformer-based coding model, using joint training optimization to obtain a base model capable of outputting fixed-dimensional embeddings, wherein the joint training includes at least: a language modeling task: tokenizing some functional annotations. The model performs masking and predicts the true value; a contrastive learning task: minimizing the positive pair distance and maximizing the negative pair distance based on positive and negative sample pairs; a classification task: supervised training on a preset classification target; wherein, the output of the base model includes species-level fixed-dimensional embedding and token-level fixed-dimensional embedding.

[0009] Further, in step S1, the sample includes a complete genome, an assembled genome, or a binned MAG; the functional annotation can incorporate genomic context information as an auxiliary feature, including proximity, collinearity, and copy number; the protein-coding sequence includes open reading frames (OFRs).

[0010] Furthermore, in step S3, the tokenization uses OG as the main sequence and KO / GO / Pfam as auxiliary channels in parallel / concatenation to enhance semantic or structural domain information; the length of the input sequence is 1024–4096, preferably 2048; the number of hidden layers in the encoding model is 8–16, preferably 10–12; and the number of heads in the multi-head attention is 8–16.

[0011] Further, in step S3, the loss function for joint training is a weighted combination of language modeling loss, contrastive loss, and classification loss, where the classification loss weight α∈[0.5, 2.0]; the training data adopts a data augmentation strategy with an augmentation ratio of 10%–40%, preferably 15%–30%; the model size and data size are selected or adjusted based on the expansion trend observed during training to obtain better validation set performance.

[0012] Furthermore, the phylogenetic tree is constructed based on the base model, including: A1: obtaining the species-level embeddings output by the base model; A2: calculating the similarity / distance between any two species embeddings, constructing a distance matrix, and denoising and normalizing the matrix; A3: reconstructing the phylogenetic tree based on the adjacency method, UPGMA, or hierarchical clustering method.

[0013] Further, species identification is performed based on the base model, including: B1: obtaining the species-level embedding of the sample to be tested using the base model; B2: constructing a TaxID prototype library using the mean of the sample embeddings of the same NCBI TaxID in the training set as the prototype vector of the TaxID; B3: calculating the distance between the sample embedding and each prototype vector, and outputting the identification result using the maximum discrimination method, wherein the maximum discrimination method is: selecting the TaxID that maximizes the difference between the candidate prototype distance and the second closest prototype distance; B4: when the minimum distance from the sample to be tested to any prototype is greater than a threshold, outputting an unknown / new species label.

[0014] Furthermore, based on the aforementioned pedestal model, metabolic gene cluster identification and key gene localization are achieved, including: C1: using OG-count sequences obtained through EggNog annotation as input; C2: fine-tuning the species-level embedding and token-level embedding of the pedestal model to achieve the identification of BGC and MGC, wherein the identification head uses sequence-cls and token-cls; C3: for class imbalance, focal loss is used for identification training; C4: predicting the product category and / or structural features of the gene cluster through multi-head output; the gene cluster includes the CRISPR-Cas system, BGC, and MGC; C5: attributing tokens based on attention map and residual correction, calculating importance scores to locate key genes / OGs in the gene cluster.

[0015] This invention also provides a base model training device based on functionally annotated abundance sequences for implementing the above-mentioned method, comprising: a data processing module for performing protein-coding sequence extraction, functional annotation, and abundance statistics / ranking, and completing tokenization; wherein the protein-coding sequence is an open reading frame; a model training module for performing joint training of Transformer encoding, language modeling / contrastive learning / classification, and outputting species-level and token-level embeddings; an application module for calculating similarity / distance based on embeddings and constructing a phylogenetic tree; performing species identification based on the TaxID prototype and maximum discrimination method, and realizing open set recognition under a threshold mechanism; identifying BGC / MGC and predicting product category / structure, and locating key genes based on attention and residual correction.

[0016] Furthermore, the data processing module supports multi-source tokenization configuration of OG main sequence + KO / GO / Pfam auxiliary channels; the application module supports denoising and normalization processing of the distance matrix.

[0017] The present invention also provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described method for training a base model based on a functional annotation abundance sequence.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. Unity and reusability: It supports multiple downstream tasks at the genome level, gene cluster level and gene / protein level in a single base model, forming a reusable representation space and reducing the cost of repeated modeling and training inference.

[0019] 2. Joint training enhances discriminative power: Through joint optimization of language modeling, contrastive learning, and classification loss, the embedding separability and downstream performance are significantly improved; under the conditions of the example, the clustering index is significantly improved (e.g., NMI is improved to 0.7059). The above values ​​are only example data and do not constitute a limitation on the scope of protection of this invention.

[0020] 3. Parameter and Data Strategy Guidelines: In the examples, optimal engineering configurations and data strategies were observed: A length of approximately 2048 inputs and a model layer count of approximately 10–12 resulted in better overall performance; 15%–30% data augmentation was superior to lower percentages; Pfam was more suitable for domain-level supplementation, while OG / KO was more suitable for species-level master characterization. These conclusions are engineering guidelines and do not constitute a limitation on the scope of this invention.

[0021] 4. Scalability (Scaling Trend): In scenarios involving large-scale data and model scaling, empirical observations show a steady improvement in performance, consistent with the scaling trend. In the 3M dataset example, relevant clustering metrics continue to improve as data and model scale (e.g., NMI increases from 0.4913 to 0.5368). The above values ​​are merely examples and do not constitute a limitation on the scope of protection of this invention.

[0022] 5. Interpretability and verifiability: Combining attention maps and residual correction to achieve token-level attribution can locate key genes (or OGs) related to specific gene clusters, which is beneficial for subsequent biological experimental verification and mechanism explanation.

[0023] 6. Open set identification and reduced database dependence: Through contrastive learning and prototype discrimination (such as the maximum discrimination method based on TaxID prototype), it has stronger robustness to unseen or highly differentiated genomes and reduces dependence on fixed reference databases.

[0024] 7. Engineering and portability: The same embedding supports applications such as phylogenetic tree construction, species identification / phenotypic prediction, BGC / MGC identification and product prediction, which facilitates modular integration and cross-task migration and deployment.

[0025] Note: The specific parameters and indicators shown in the examples are representative experimental results used to illustrate the technical effects of the present invention and do not constitute a limitation on the scope of protection of the present invention. Attached Figure Description

[0026] Figure 1 is a schematic diagram of the Ortholog Group (OG) in this invention, used to explain the definition of OG, which is a group of homologous genes derived from a single gene of the most recent common ancestor of all studied species; Figure 2 is a schematic diagram of the overall structure of the Orthoformer base model of this invention, including input tokenization, embedding layer, Transformer encoder, multi-task training head (language modeling, contrastive learning, classifier) ​​and other modules; Figure 3 is a block diagram of the device modules of this invention, including data processing module, model training module and downstream application module; Figure 4 is a flowchart of the microbial species identification based on the base model of this invention, showing the entire process of input sample functional annotation, tokenization, base model embedding, comparison with TaxID prototype, and maximum discrimination method; Figure 5 is a flowchart of the phylogenetic tree construction based on the base model of this invention; Figure 6 is a flowchart of the gene cluster and key gene identification based on the base model of this invention. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0028] This invention provides a base model training method based on functional annotation abundance sequences, comprising the following steps: S1, Data Processing: Using a species genome or metagenomics as a sample, bioinformatics analyses such as sequence homology search, domain identification, or evolutionary classification are performed on its protein-coding sequences to obtain corresponding functional annotations. Figure 1 shows a schematic diagram of the Ortholog Group (OG) in this invention, used to explain the definition of OG, which is a complete group of homologous genes derived from a single gene of the most recent common ancestor of all studied species; the sample includes a complete genome, an assembled genome, or a binned MAG; the functional annotation can be combined with genomic context information (proximity / collinearity / copy number) as auxiliary features, wherein: proximity (gene neighborhood): refers to the relative positional relationship of the target gene with neighboring genes or functional modules in the genomic sequence; collinearity (synteny): refers to the conservation of gene arrangement order or gene blocks between different genomes; copy number variation (copy number variation): refers to the difference in the number of copies of a gene or genomic region in the same sample or different samples.

[0029] S2. Abundance sorting: Sort the data after processing in step S1 in descending order of abundance to generate a sample abundance sequence.

[0030] S3. Model Training: The abundance sequence described in step S2 is tokenized and input into the Transformer-based base model. Figure 2 shows a schematic diagram of the overall structure of the Orthoformer base model of this invention, including modules such as input tokenization, embedding layer, Transformer encoder, and multi-task training head (language modeling, contrastive learning, classifier). Fixed-dimensional embeddings are obtained through multi-task joint training (language modeling, contrastive learning, classification loss), including both species-level embeddings and token-level embeddings. Figure 3 is a block diagram of the device modules of this invention, including a data processing module, a model training module, and a downstream application module.

[0031] It should be noted that the loss function for joint training is a weighted combination of language modeling loss, contrastive loss, and classification loss, where the classification loss weight α∈[0.5, 2.0]. The model size and data size are selected or adjusted based on the expansion trends observed during training to obtain better validation set performance.

[0032] S4. Downstream tasks: Use the embedding results to complete tasks at different levels: genome level, gene cluster level, and gene / protein level.

[0033] Furthermore, in the data processing step S1, the genome or metagenomics includes binned MAG and other genomes; the protein-coding sequences include open reading frames (OFRs); and the bioinformatics functional annotations include ortholog group (OG), gene ontology (GO), KEGG ontology (KO), Pfam, etc.

[0034] Furthermore, in step S2, abundance ranking, the abundance is defined as the number of times each functional annotation appears in the sample. Preferably, weighted occurrence counts (e.g., by gene length, coverage, confidence weight) or standardized counts (e.g., CPM / TPM, in-sample normalization) can be used as alternative implementations to adapt to different data distributions. Furthermore, the generated functional annotation abundance sequence retains a stable order of descending order and parallel items to maintain cross-sample comparability.

[0035] Further, step S3, model training, includes tokenizing the functional annotation abundance sequence and inputting it into a Transformer-based encoding model, employing a multi-task joint training strategy, wherein: language modeling task: for the functional annotation abundance sequence, tokenize a portion of the functional annotations. Masking and predicting the true values ​​enable the model to learn the biological contextual relationships between annotations (e.g., co-occurrence patterns and functional collaborations of metabolic pathway-related annotations), providing functional association features to support subsequent tasks such as metabolic gene cluster (MGC) identification. For contrastive learning tasks, positive sample pairs (functional annotation abundance sequences of different strains of the same species or the same type of functional gene cluster) and negative sample pairs (e.g., functional annotation abundance sequences of closely related species or different types of functional gene clusters) are constructed. The contrastive loss function is used to optimize model parameters, making the embedding distance of similar samples closer and the embedding distance of different samples farther, thus strengthening the species / function specificity of the embedding vectors and overcoming the bottleneck of identifying new species and closely related species. For classification loss, for coarse-grained basic tasks such as phylum / class classification and gene cluster (BGC / MGC) type classification, the cross-entropy loss function is used to optimize parameters based on the model output of the functional annotation abundance sequences, enabling the embedding vectors to have classification discrimination capabilities. This provides a foundation for downstream fine-grained tasks (e.g., species-level identification) and avoids redundant modeling.

[0036] Optionally, the loss weights of the joint training can be dynamically adjusted according to the importance of the task or a gridded search can be performed within a preset range; the model output includes two types of representations: species-level fixed-dimensional embeddings and token-level fixed-dimensional embeddings.

[0037] Furthermore, the downstream task in step S4 unfolds at multiple levels based on the embedding representation output by the base model, including genome-level tasks, gene cluster-level tasks, and gene / protein-level tasks: Genome-level tasks: a) Phylogenetic tree construction: Based on species-level fixed-dimensional embedding integrating single-copy orthologous genes and multi-copy genes, paralogous genes, gene family expansion / contraction, and functional and genomic organization signals, pairwise similarity / distance between species is calculated and a distance matrix is ​​constructed. A tree structure of approximate species phylogenetic relationships is obtained using distance methods or hierarchical clustering strategies (such as NJ, UPGMA), which differs from the traditional reconstruction method that relies on single-copy orthologous gene alignment and replacement models. Figure 5 is a flowchart of the phylogenetic tree construction based on the base model of this invention; b) Species identification: Species-level embeddings obtained by pre-training on large-scale multi-species genomes using a base model are combined with a classifier or contrastive learning framework to determine the species of the input genome at multiple classification levels. For novel or highly differentiated genomes lacking annotation, it still has high identification potential. Figure 4 is a flowchart of the microbial species identification based on the base model of this invention, showing the entire process of input sample functional annotation, tokenization, base model embedding, comparison with TaxID prototype, and maximum discrimination method; c) Phenotypic prediction: Phenotypic classification / regression models are trained on the basis of species-level embeddings to achieve prediction of features including but not limited to pathogenicity and drug resistance phenotypes.

[0038] Gene cluster level tasks: a) CRISPR-Cas system identification: Based on the functional association signals learned by the pedestal model, sequence labeling (token-cls) fine-tuning is performed to identify and locate CRISPR-related genes and distinguish different types of Cas systems. Figure 6 is a flowchart of gene cluster and key gene identification based on the pedestal model of this invention; b) BGC / MGC identification and product prediction: Combining species-level embedding and token-level embedding, sequence-cls and token-cls fine-tuning are performed to identify BGC and MGC in the microbial genome and output product category and / or structural feature prediction.

[0039] Gene / protein level tasks: a) Key gene localization: Based on the pre-trained classification / recognition model, interpretability attribution is performed by combining attention maps and residual correction, token importance scores are calculated, and key genes or OGs highly related to specific gene clusters are identified; b) Mining of special environmental functions: For microorganisms in extreme environments, candidate protein families that may confer phenotypes such as high salt tolerance, high temperature tolerance, and radiation tolerance are screened by analyzing OGs with high attention weights for exploration of biotechnology applications.

[0040] Referring to Figure 3, the present invention also provides an apparatus for implementing the above method, comprising: a data processing module: performing ORF extraction, functional annotation, and abundance statistics; a model training module: performing tokenization, Transformer encoding, multi-task joint training, and embedding generation; and a downstream task module (application module), including: a phylogenetic tree construction submodule (similarity / distance calculation, matrix normalization, NJ / UPGMA); a species identification and phenotypic prediction submodule (TaxID prototype library maintenance, maximum discrimination, phenotypic classifier); and a gene cluster identification and key gene localization submodule (sequence-cls, token-cls, multi-head output, attention + residual correction attribution).

[0041] The above modules can be implemented on the same computing device or in a distributed deployment manner; or the program instructions can be stored on a computer-readable storage medium and the steps can be completed when executed by a processor.

[0042] Example 1: Data Sources and Preprocessing 1. Data Sources and Sample Units This example uses a single species genome or metagenomic binned MAG as the sample unit. Exemplary sources include: Genome classification datasets (such as high-quality GTDB sets): used for species-level pre-training and species identification / phenotype prediction; Functional gene cluster annotation data (such as antiSMASH, gutSMASH running on 123K genomes): used for BGC / MGC identification and product prediction; NCBI Taxonomy: used to establish TaxID hierarchies and prototype libraries (centroid sets).

[0043] Note: The above data sources are merely feasible examples and do not constitute a limitation of the present invention.

[0044] 2. ORF Extraction and Functional Annotation: Open reading frames (ORFs) are extracted from each sample and bioinformatics functional annotations are performed. Annotation types include at least: OG (Ortholog Group): representing a group of orthologous genes derived from a common ancestor gene, reflecting gene family expansion / contraction; GO (Gene Ontology): ontology tags reflecting gene function and pathways; KO (KEGG Ontology): mapping genes to pathways / modules / brites, revealing metabolic networks and signal regulation; Pfam: family tags at the protein domain level.

[0045] Note: OG / KO / GO / Pfam are classic annotation information. This invention may choose one or more combinations of them as the source of subsequent tokens.

[0046] 3. Abundance Statistics, Sorting, and Sample Sequence Generation: The frequency of occurrence (i.e., abundance) of each functional annotation within the sample is statistically analyzed to obtain a vector representation; the functional annotation abundance sequence of the sample is obtained by sorting the abundance in descending order, which serves as the basis for the input sequence of the language model.

[0047] Optional: Use weighted occurrence counts (e.g., weighted by gene length, coverage, or annotation confidence) or in-sample normalization (e.g., CPM / TPM) to accommodate different sequencing depths.

[0048] Parallel items maintain a stable order to ensure comparability across samples.

[0049] 4. Lexicon and tokenization map functional annotations to lexicon IDs; supports: Main channel: OG as the main sequence (token), preferred for species-level representation; Secondary channel: KO / GO / Pfam tokens can be concatenated or introduced in parallel to enhance semantic diversity and structural domain information; Special tokens: <mask> 、 <cls> 、 <pad>Low-frequency / OOV tokens employ a merging bucket strategy; positional encoding / weighted encoding may be included (e.g., embedding abundance or confidence into the token vector or as a parallel feature).

[0050] 5. Data cleaning and partitioning: low-confidence annotation and outlier filtering; training / validation / test sets are stratified by species to avoid information leakage caused by near-repetition; when there is severe class imbalance, resampling or loss reweighting is performed on minority classes.

[0051] Example 2: Orthoformer Model Training and Parameter / Data Strategy. This example uses a Transformer-based encoding model, inputting tokenized functional annotation abundance sequences. Joint training employs language modeling, contrastive learning, and classification loss to output species-level and token-level fixed-dimensional embeddings. To verify the universality of the engineering configuration and strategy, the following seven sub-experiments were conducted: Sub-experiment 2-1: Attention Layer and Head Number Design: Fixed max_len=2048, hidden=512, contrasting layer / head combinations.

[0052] As shown in Table 2-1 below, Max Length represents the maximum input length; Hidden Size represents the intermediate layer dimension; Hidden Layers represents the number of layers; Val Loss represents the validation set loss; Test Loss represents the test set loss; NMINMI (Normalized Mutual Information) measures the degree of overlap between the clustering results and the true labels (from an information theory perspective); ARI (Adjusted Rand Index) measures whether the clustering results are correctly grouped (whether they are in the same group as the true labels); ASW (Average Silhouette Width) measures the quality of the clustering structure itself (independent of the true labels).

[0053] Results (Table 2-1) Conclusion: Under the conditions of this embodiment, moderately increasing the number of layers is better than increasing the number of heads, resulting in more stable convergence and better parameter efficiency.

[0054] Sub-experiment 2-2: Maximum input length design: max_len∈{512, 2048, 4000}. Results (Table 2-2) Conclusion: 2048 is the optimal overall value; excessively long sequences present optimization difficulties and information dilution.

[0055] Sub-experiment 2-3: Data normalization results (Table 2-3) Conclusion: In this task scenario, normalization did not bring any gain; in fact, the embedding resolution decreased.

[0056] Sub-experiment 2-4: Data augmentation results (Table 2-4) Conclusion: 15%–30% enhancement is better than 5%–15%, and generalization is better.

[0057] Sub-experiment 2-5: Adding classification loss results (Table 2-5) Conclusion: Joint classification loss significantly improves separability (e.g., NMI≈0.706, ARI≈0.160), which is an unexpected technical effect (the values ​​are for illustrative purposes only and do not limit the scope of protection).

[0058] Sub-experiment 2-6: Comparison results of Pfam annotations (Table 2-6) Conclusion: Pfam is more suitable for domain supplementation; OG / KO is suitable as a species-level master characterization.

[0059] Sub-experiment 2-7: Scaling trend (3M) results (Table 2-7) Conclusion: As the scale of data and models increases, the metrics steadily improve, showing scalability (scaling trend).

[0060] Evaluation metrics: Clustering metrics use NMI / ARI / ASW, and validation / test set loss is reported; downstream tasks use classification / recognition metrics that match the task. Computing environment: GPU model and memory, main framework and optimizer, learning rate, batch size, epochs, etc., are as shown in the examples without constituting limitations.

[0061] Example 3: Construction of Phylogenetic Trees Based on Embeddings 1. Method Flow Taking the species-level embeddings output by the trained pedigree model as input, calculate the pairwise cosine / Euclidean distance between species; denoise and normalize the distance matrix (e.g., truncate by quantiles and standardize rows and columns); generate phylogenetic trees using adjacency join (NJ), UPGMA, or hierarchical clustering.

[0062] 2. Key technical features include: explicit / implicit integration of signals such as single-copy orthologous, multiple-copy, paralogous, family expansion / contraction, and genome organization; no strict MSA and substitution model required, robust to gene deletion / redundancy; topological comparison can be performed using RF distance / quadruple consistency assessment and 16S / SCG trees (bootstrapping optional).

[0063] Example 4: Species Identification and Phenotypic Prediction 1. Species Identification (Maximum Separation Method) Constructing a TaxID Prototype Library: Take the mean vector of the sample embeddings of the same NCBI TaxID in the training set as the prototype of the TaxID and construct a TaxID prototype library; Inference: Calculate the distance between the query sample embedding and each prototype vector, and use the maximum separation method to select the TaxID with the largest distance advantage compared with other prototypes as the output; Open Set: Set a confidence threshold / distance boundary. When the sample does not meet the threshold with any prototype (when it is greater than the threshold), output the label "Unknown / New Species".

[0064] Note: This comparative learning paradigm can draw on the ideas of CLEAN-like methods, but this invention focuses on the identification of TaxID-level and open set recognition.

[0065] 2. Phenotypic prediction: Train lightweight models (such as Logistic / MLP) on species-level embeddings to predict phenotypes such as pathogenicity and drug resistance; cross-entropy or weighted loss can be used according to task characteristics to alleviate class imbalance.

[0066] 3. Evaluation and comparison with baseline methods such as 16S / ANI / k-mer deep models: Top-1 / Top-k, AUROC, AUPRC; emphasizing robustness and reduced database dependence in new species / closely related species scenarios.

[0067] Conclusion: Compared with methods such as 16S / ANI, this method is more robust to unseen or highly differentiated genomes and has reduced database dependence.

[0068] Example 5: Gene Cluster Identification and Key Gene Localization 1. Data and Labeling The OG-count sequences obtained by EggNog annotation are used as input; for example, taking the 123K genome as an example, antiSMASH and gutSMASH are run to obtain the location, product category and structural information of BGC / MGC regions; OG sequences are obtained by EggNog annotation of all samples; BGC / MGC tags are aligned to form a supervised sample set of OG-count tokenizer.

[0069] 2. Training and Inference of BGC / MGC Recognition: Combining species-level embedding and token-level embedding of the base model for sequence-cls + token-cls fine-tuning to achieve the recognition of BGC and MGC; Loss Function: To address class imbalance, focal loss is adopted (example γ∈[1,3], α∈[0.25,0.75], without limitation); Product Prediction: Multi-head output of product category and (optional) structural fingerprint / substructure set; CRISPR-Cas: Locating Cas-related genes and distinguishing types using sequence labeling (token-cls).

[0070] 3. Interpretability and Key Genes: Attribution of tokens is performed based on attention map + residual correction, and importance scores are calculated. Thresholds / rankings are set to determine key OGs / genes, and the site intervals are output for subsequent experimental verification.

[0071] 4. Evaluation and baseline comparison with tools such as ClusterFinder and DeepBGC: F1 score, precision / recall, mAP, and AUROC; showcasing typical novel BGC / MGC discovery cases and key gene interpretations.

[0072] Conclusion: The unified framework covers both BGC and MGC, and can locate key genes and support biological validation.

[0073] General Evaluation Instructions (Applicable to all embodiments): Metrics: Clustering (NMI / ARI / ASW), classification (Accuracy / Top-k / AUROC / AUPRC), sequence labeling (Token-F1), and detection (mAP) are defined separately; Statistical significance: Repeated random seeding, reporting mean ± standard deviation; t-test or nonparametric test is performed on key differences; Computational environment: GPU / CPU model, video memory, and main software stack; Reproducibility: Fixed random seed, providing data partitioning and evaluation script instructions.

[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0075] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.< / pad> < / cls> < / mask>

Claims

1. A base model training method based on functional annotation abundance sequences, characterized in that, include: S1: Perform bioinformatics functional annotation on protein-coding sequences of species genomes or metagenomic samples; the functional annotation includes one or more of OG, GO, KO, and Pfam, where OG represents a group of orthologous genes derived from a common ancestor gene, reflecting gene family expansion / contraction; GO is an ontology label reflecting gene function and pathways; KO is used to reveal metabolic networks and signal regulation; Pfam represents a family label at the protein domain level; S2: Count the occurrence frequency of the functional annotations in the sample and arrange them in descending order of abundance to construct the functional annotation abundance sequence of the sample; S3: Tokenize the abundance sequence and input it into a Transformer-based coding model, and use joint training to optimize and obtain a base model that can output fixed-dimensional embeddings. The joint training includes at least: language modeling task: masking some functional annotation tokens and predicting their true values; contrastive learning task: minimizing the positive pair distance and maximizing the negative pair distance based on positive and negative sample pairs; classification task: supervised training on a preset classification target; wherein, the output of the base model includes species-level fixed-dimensional embeddings and token-level fixed-dimensional embeddings.

2. The base model training method based on functional annotation abundance sequence according to claim 1, characterized in that, In step S1, the sample includes a complete genome, an assembled genome, or a binned MAG; the functional annotation can incorporate genomic context information as an auxiliary feature, which includes proximity, collinearity, and copy number; the protein-coding sequence includes open reading frames (OFRs).

3. The base model training method based on functional annotation abundance sequence according to claim 1, characterized in that, In step S3, the tokenization uses OG as the main sequence and KO / GO / Pfam as auxiliary channels in parallel / concatenated order to enhance semantic or structural domain information; the length of the input sequence is 1024–4096, preferably 2048; the number of hidden layers in the encoding model is 8–16, preferably 10–12; and the number of heads in the multi-head attention is 8–16.

4. The base model training method based on functional annotation abundance sequence according to claim 1, characterized in that, In step S3, the loss function for joint training is a weighted combination of language modeling loss, contrastive loss, and classification loss, where the classification loss weight α∈[0.5, 2.0]; the training data adopts a data augmentation strategy with an augmentation ratio of 10%–40%, preferably 15%–30%; the model size and data size are selected or adjusted based on the expansion trend observed during training to obtain better validation set performance.

5. The base model training method based on functional annotation abundance sequence according to claim 1, characterized in that, The phylogenetic tree is constructed based on the base model, including: A1: obtaining the species-level embeddings output by the base model; A2: calculating the similarity / distance between any two species embeddings, constructing a distance matrix, and denoising and normalizing the matrix; A3: reconstructing the phylogenetic tree based on the adjacency method, UPGMA, or hierarchical clustering method.

6. The base model training method based on functional annotation abundance sequence according to claim 1, characterized in that, Species identification based on the aforementioned base model includes: B1: obtaining the species-level embedding of the sample to be tested using the base model; B2: constructing a TaxID prototype library using the mean of the sample embeddings of the same NCBI TaxID in the training set as the prototype vector of that TaxID; B3: calculating the distance between the sample embedding and each prototype vector, and outputting the identification result using the maximum discrimination method, wherein the maximum discrimination method is: selecting the TaxID that maximizes the difference between the candidate prototype distance and the second closest prototype distance; B4: outputting an unknown / new species label when the minimum distance from the sample to be tested to any prototype is greater than a threshold.

7. The base model training method based on functional annotation abundance sequence according to claim 1, characterized in that, The identification of metabolic gene clusters and the localization of key genes are accomplished based on the aforementioned base model, including: C1: using OG-count sequences obtained through EggNog annotation as input; C2: fine-tuning the species-level embedding and token-level embedding of the base model to identify BGC and MGC, wherein the recognition head uses sequence-cls and token-cls; C3: using focal loss for recognition training to address class imbalance; C4: predicting the product category and / or structural features of the gene clusters through multi-head output; the gene clusters include the CRISPR-Cas system, BGC, and MGC; C5: attributing tokens based on attention map and residual correction, calculating importance scores to locate key genes / OGs in the gene clusters.

8. A pedestal model training apparatus based on functional annotation abundance sequence for implementing the method as described in claim 1, characterized in that, include: The data processing module is used to perform protein-coding sequence extraction, functional annotation, abundance statistics / sorting, and tokenization; the protein-coding sequence is an open reading frame. The model training module is used to perform joint training of Transformer encoding, language modeling / contrastive learning / classification and output species-level and token-level embeddings; The application module is used to calculate similarity / distance based on embedding and construct phylogenetic trees; identify species based on the TaxID prototype and maximum discrimination method, and realize open set identification under the threshold mechanism; identify BGC / MGC and predict product category / structure, and locate key genes based on attention and residual correction.

9. The base model training device based on functional annotation abundance sequence according to claim 8, characterized in that, The data processing module supports multi-source tokenization configuration of OG main sequence + KO / GO / Pfam auxiliary channels; the application module supports denoising and normalization of the distance matrix.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the pedestal model training method based on functional annotation abundance sequences as described in any one of claims 1-7.

Images