Genome prediction model based on attention weight distribution and adaptive residual learning

Abstract

The invention discloses a genome prediction model based on attention weight distribution and adaptive residual learning, and belongs to the technical field of genome selection. The method comprises the following technical steps: acquiring a 012 coding matrix of genetic polymorphism sites, calculating the weight of each SNP site by using a multi-layer perceptron (MLP), combining input features with the weights through an adaptive residual learning mechanism, and further performing fitting in a full-connection neural network; and after model training is completed, outputting a determination coefficient (R2) of the test set, a Pearson's correlation coefficient (PCC) and a weight file of each SNP site. The innovation point of the method is that an attention mechanism is introduced, the weight of the SNP site is dynamically distributed, the influence of the weight on phenotype is analyzed from the global perspective, and adaptive residual learning is combined. Automation from SNP data to phenotype prediction is achieved through an SNPWhighNet.py program, and the method has the advantages of being high in precision and high in interpretability.

Description

Technical Field

[0001] The present invention belongs to the technical field of genome selection, and in particular relates to a genome prediction model based on attention weight allocation and adaptive residual learning. Background Art

[0002] Genomic prediction (GP), also known as genomic selection (GS), is an advanced method for predicting complex traits using genome-wide genetic markers. This method was first proposed by Meuwissen, TH, (Hayes, BJ, & Goddard, ME (2001). Prediction of total genetic value using genome-wide dense marker maps. Genetics, 157 (4), 1819-1829.) to address the limitations of traditional marker-assisted selection (MAS). This method mainly uses high-throughput sequencing technology to obtain molecular markers across the entire genome (such as hundreds of thousands to millions of SNP sites) and construct a genotype-phenotype association dataset. Through machine learning or statistical models (such as GBLUP, Bayesian methods, and deep learning), a predictive relationship between genotype data and target phenotypes (such as crop yield and disease resistance) is established. Then, in the early stages of individual growth and development (such as the seedling stage), the performance of adult traits can be predicted based solely on genotype data, without waiting for the phenotype to fully manifest, thereby shortening the breeding cycle.

[0003] However, the application of GP faces significant challenges. First, phenotypic data are usually limited (n individuals), while genotypic data involve a large number of markers (p markers), resulting in data dimension mismatch (p>>n). Second, non-additive effects between genes (such as dominance, epistasis, etc.) increase the complexity of prediction, and traditional linear and nonlinear algorithms are difficult to effectively capture these complex relationships (de los Campos, G., Hickey, JM, Pong-Wong, R., Daetwyler, HD, & Calus, MP (2013). Whole-genome regression and prediction methods applied to plant and animal breeding. Genetics, 193 (2), 327-345.). In recent years, neural networks have gradually been introduced into the field of GP, but the traditional multi-layer perceptron (MLP) model has not significantly outperformed the traditional linear model method and machine learning method in terms of prediction accuracy (Montesinos-López, O.A., Montesinos-López, A., Pérez-Rodríguez, P., de los Campos, G., Eskridge, K., & Crossa, J. (2019). A comparative study of machine learning methods for genomic prediction. Frontiers in Genetics, 10, 1071.).

[0004] With breakthroughs in algorithms, attention mechanisms have achieved remarkable results in fields such as natural language processing (NLP), effectively capturing long-range dependencies in sequence data (Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, AN, ... & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.). Furthermore, the application of residual connections in deep neural networks alleviates the vanishing gradient problem and enhances the training stability of deep networks (He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770-778.). However, the application of attention mechanisms in genomic selection remains underdeveloped. Traditional attention mechanisms rely on explicit key-value (KQV) calculations, but the SNP matrix lacks the spatial relationships and encoding structures found in natural language, making it difficult to directly transfer (Zhou, J., Troyanskaya, O.G. (2018). Predicting effects of noncoding variants with deep learning-based sequence model. Nature Methods, 15(5), 361-364.). In addition, the high dimensionality and sparsity of SNP data further increase the difficulty of applying the attention mechanism in this field (Zhang, Z., Zhao, Y., & Liu, J. (2019). Deep learning for genomic data: A comprehensive review. Briefings in Bioinformatics, 20(4), 1239-1252.).Although some studies have attempted to introduce deep learning models such as convolutional neural networks (CNN) into genomic prediction, such as DNNGP (Deep Neural Network for Genomic Prediction) and DeepGS (Deep Learning for Genomic Selection), these models mainly rely on local receptive fields to extract features and are unable to effectively capture the global dependencies in SNP data, limiting the further improvement of prediction accuracy (Bellot, P., de los Campos, G., & Pérez-Enciso, M. (2018). Can deep learning improve genomic prediction of complex traits? Genetics, 210(3), 809-819.; Ma, W., Qiu, Z., Song, J., Li, J., Cheng, Q., & Zhai, J. (2020). DNNGP: A deep neural network-based method for genomic prediction using multi-omics data.Bioinformatics,36(1),260-268.;Wang,X.,Xu,Y.,&Li,J.(2021).DeepGS:Adeep learning framework for genomic selection.BMC Genomics,22(1),1-12.). Therefore, how to effectively combine the attention mechanism and residual connection in genomic selection remains a technical challenge that needs to be solved urgently. Summary of the Invention

[0005] The purpose of the present invention is to provide a genome prediction model based on attention weight allocation and adaptive residual learning to solve the problem of accurately predicting phenotype from genotype.

[0006] The present invention achieves the above-mentioned object through the following technical solution: providing a genome prediction model based on attention weight allocation and adaptive residual learning, which is constructed by the following steps:

[0007] (1) Data preparation and preprocessing: Obtain the genotype file and phenotype file of the target species. Genotype files are usually stored in VCF (Variant Call Format) files, a universal genetic variation storage format. This file contains the variation information of each site of each individual. At the same time, the phenotypic data related to the target trait must also be obtained to ensure that the phenotypic data corresponds one-to-one with the individuals in the VCF file. Use the Plink tool to remove SNP sites with MAF below the threshold, convert the VCF file into binary format, and use its LD filtering function to retain SNP sites with higher independence. Use the Plink tool to convert the filtered SNP sites into a 012 matrix, where 0, 1, and 2 represent the homozygous reference type, heterozygous type, and homozygous variant genotype, respectively. This matrix is ​​used as the input data of the model (stored in TSV format).

[0008] (2) Model construction: Based on the PyTorch framework, a deep learning model is constructed, with the modified attention module and regressor as the core. The attention module implicitly calculates the weight of each SNP site through a multi-layer fully connected network (MLP) and the softmax function of the output layer, avoiding explicit key-value calculations and thus reducing computational complexity. The hidden layer of the multi-layer fully connected network uses the GELU activation function. The module also includes an adaptive residual learning mechanism: the original input features and the features weighted by the attention weights are fused through the formula F = x × (weights) + x to alleviate the gradient vanishing problem, where x is the SNP data input to the deep learning model, weights is the attention weight generated by the attention module, and F is the input of multiple fully connected layers of the regressor module. The weighted SNP data is input to the regressor module, which uses a multi-layer fully connected network and uses the GELU activation function and Dropout layer between layers to improve the generalization ability of the model and prevent overfitting. Preferably, layer normalization (LayerNorm) is introduced between multiple fully connected layers of the regressor module to improve the model convergence speed and training stability, and finally generate the predicted value of the target trait in the output layer.

[0009] During the forward propagation process, the input SNP matrix passes through the attention module to generate attention weights and weight the input features. The weighted features output the predicted value of the target trait through the regressor, and the attention weights are returned at the same time for subsequent analysis of the impact of the SNP site on the prediction results.

[0010] (3) Model training and evaluation: The dataset is divided into training set, validation set and test set. The mean square error (MSE) is used as the loss function and the Adam optimizer is used for training. An early stopping mechanism is introduced to prevent overfitting and the model performance is evaluated on the test set. The coefficient of determination (R 2) and Pearson correlation coefficient (PCC) as evaluation indicators, outputting information on changes in loss function, coefficient of determination, and Pearson correlation coefficient with training rounds, as well as dynamic weight output of each SNP site, to explain the impact of each SNP site on the predicted phenotype.

[0011] The beneficial effects of the present invention are as follows:

[0012] (1) Attention mechanism with implicit weight calculation: Traditional attention mechanism models (such as Transformer) implement weight allocation through explicit key-value calculation (K, Q, V). The present invention implicitly calculates the weight relationship of each SNP site through multiple fully connected layers and softmax functions, realizing an attention mechanism without explicit key-value calculation. This method not only reduces the computational complexity of the model, but also makes the weight allocation process more efficient and flexible. At the same time, these dynamic weights will be updated in real time with the model, and the optimal weight of each SNP site under the corresponding phenotype can be calculated, providing potential profiling information for the importance analysis of gene sites.

[0013] (2) Capturing the global receptive field: Compared with convolutional neural networks (CNNs), existing genomic prediction models (such as DNNGP, DeepGS, etc.) rely on the local receptive field of CNN to extract features. However, gene interactions in SNP data may be global, spanning multiple non-locally associated sites. It is difficult for local receptive fields to directly establish long-distance dependencies, which may lead to the neglect of global interactions. The present invention adopts an attention-like mechanism that can effectively capture the global dependencies in SNP data, making up for the limitations of CNN in genomic prediction.

[0014] (3) Information preservation through residual connections: In traditional models, the training data input to the network is the original data (x) of the SNP. However, after calculating the weights through the attention mechanism, the present invention uses residual connections (F = x * weights + x) to fuse the original data with the weighted features. This design ensures that even when the attention module adjusts the weights of the SNP, the original information is still retained, avoiding the loss of important features during the feature transformation process. Although the output data range is [0, 2x], it performs well in actual operation, further improving the robustness and prediction accuracy of the model.

[0015] In summary, the present invention significantly improves the accuracy and efficiency of genome prediction through implicit weight calculation, global receptive field capture, and information retention of residual connections, providing strong technical support for biological breeding and genomics research. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the SNPWeightNet model architecture;

[0017] Figure 2 This is the prediction result of SNPWeightNet for four different cotton agronomic traits, and other algorithms are used as controls. DETAILED DESCRIPTION

[0018] The following describes in detail the embodiments, features, and aspects of the present invention with reference to the accompanying drawings, but does not limit the present invention. Any embodiment expanded from the embodiments of the present invention and all other embodiments obtained by persons of ordinary skill in the art without inventive effort are within the scope of protection of the present invention.

[0019] Example 1:

[0020] 1. Processing of genotype and phenotype files of cotton samples.

[0021] (1) Filter low-frequency SNP sites with a minimum allele frequency (MAF) less than 5% and generate a binary file. The execution command is as follows:

[0022] plink --bfile <input> --chr-set 12--allow-extra-chr--maf 0.05--make-bed--out <output>

[0023] (2) Perform linkage disequilibrium (LD) filtering to remove redundant SNP sites in high LD regions, such as r 2 The threshold is 0.1, the sliding window size is 50, and it moves 5 sites at a time. The output of the previous step is used as the input of this step:

[0024] plink --bfile <input> --indep-pairwise 50 5 0.1--allow-extra-chr--out <output>

[0025] (3) Extract the SNP sites after LD filtering and convert them into 012 matrix format. Use the output of the previous step (2) as the input of --extract in this step. Use the input of the previous step (1) as the input of --bfile.

[0026] plink --bfile <input> --extract <input> --recode A--out <output>

[0027] (4) Convert the .raw file generated by PLINK into TSV format as the SNP data file, and prepare the phenotype file corresponding to the SNP data.

[0028] 2. Model construction and training.

[0029] (1) Build a deep learning model based on the PyTorch framework, such as Figure 1 As shown in Figure 2, the core of the model consists of an improved attention module and a revised regressor module. The attention module implicitly calculates the weight of each SNP locus through a multi-layer fully connected network (MLP) and a softmax function. The regressor module uses multiple fully connected layers, combined with the GELU activation function, dropout layers, and layer normalization (LayerNorm) to enhance generalization capabilities, and retains the original information through adaptive residual connections. The mean squared error (MSE) is used as the loss function, and the Adam optimizer is used for parameter optimization. An early stopping strategy is introduced to prevent overfitting.

[0030] (2) The SNP data and phenotypic data obtained in step 1 are used as input and run in the SNPWeightNet.py program. Based on the above model framework, this program fits a model for predicting phenotypic values ​​based on SNP data and outputs evaluation indicators of the model on the test set, including the coefficient of determination (R2) and the Pearson correlation coefficient (PCC). At the same time, it calculates and generates images of the loss function, the coefficient of determination, and the Pearson correlation coefficient as a function of training rounds (epochs), as well as a weight file for each SNP site.

[0031] The deep learning model of this embodiment was used to predict the traits of four different cottons, and the model performance was compared with other algorithms (artificial neural network, K nearest neighbor, xgboost, random forest, GBLUP, DNNGP). Figure 2 It can be seen that the model performance of the present invention (SNPWeightNet) is the best, with the predicted PCCs of fiber length (FibLen), fiber strength (FibStr), fiber micronaire value (FibMic) and fiber elongation (FibElo) reaching 0.79, 0.77, 0.77 and 0.71, respectively, which are more than 10% higher than the benchmark GBLUP.

[0032] It should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made without inventive effort shall fall within the scope of protection of the present invention.< / output> < / output> < / output>

Claims

1. A genome prediction model based on attention weight allocation and adaptive residual learning, characterized by: It is constructed by the following steps: Obtain the genotype file and phenotype file of the target species; for the genotype file, remove SNP sites with MAF below the threshold, perform linkage disequilibrium filtering, and generate a 012 encoding matrix based on the screened SNP sites as the SNP data file; A deep learning model is constructed and trained based on the SNP data file and the phenotype file; the attention module of the deep learning model is used to assign attention weights to SNP sites and generate weighted SNP data. The attention module adopts a multi-layer fully connected network, and the hidden layer of the multi-layer fully connected network adopts a GELU activation function. The weighted SNP data is input into a regressor module. The regressor module adopts multiple fully connected layers and uses a GELU activation function and a Dropout layer between layers to finally output a phenotypic prediction value.

2. The genome prediction model according to claim 1, characterized in that The genotype file of the target species is stored in VCF format, and a binary file is generated after removing SNP sites with MAF below the threshold. The SNP data file is stored in TSV format.

3. The genome prediction model according to claim 1, characterized in that The SNP sites with MAF lower than the threshold were removed, linkage disequilibrium filtering was performed, and a 012 coding matrix was generated based on the screened SNP sites, all using the plink tool.

4. The genome prediction model according to claim 1, characterized in that The deep learning model also includes an adaptive residual learning mechanism, specifically: combining input features with attention weights through the formula F = x × weights + x, where x is the SNP data input to the deep learning model, weights is the attention weight generated by the attention module, and F serves as the input to multiple fully connected layers of the regressor module.

5. The genome prediction model according to claim 1, characterized in that The multiple fully connected layers of the regressor module also include layer normalization.

6. The genome prediction model according to claim 1, characterized in that The training is performed based on the SNP data file and the phenotype file, using mean square error as the loss function and Adam optimizer for parameter optimization.

7. The genome prediction model according to claim 1, characterized in that The output of the deep learning model includes: phenotypic prediction value; determination coefficient and Pearson correlation coefficient obtained by testing the trained deep learning model, which are used to evaluate the model's prediction effect; dynamic weight output of each SNP site, which is used to explain the impact of each SNP site on the predicted phenotype.

Images