GATE intelligent prediction method based on gating attention mechanism

Abstract

The invention relates to a GATE intelligent prediction method based on a gated attention mechanism, and belongs to the technical field of plant phenotype prediction. The method comprises the following steps: preprocessing transcriptome data; extracting local co-expression features among genes through multi-channel one-dimensional convolution; an important feature channel is weighted in a self-adaptive mode through a channel attention module; key features are dynamically screened and enhanced through a gating mechanism; performing feature fusion and stable training in combination with residual connection and layer normalization; and finally carrying out regression or classification prediction. The method provided by the invention can efficiently capture a nonlinear regulation relationship between genes, improves prediction precision and model interpretation ability, is suitable for transcriptome-driven phenotype prediction of multiple species and multiple traits, and can assist in identifying potential regulation genes.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of plant phenotype prediction, and particularly relates to a GATE intelligent prediction method based on a gated attention mechanism. BACKGROUND

[0002] With the continuous decline in the cost of high-throughput sequencing, gene expression can be accurately quantified at the whole genome scale, making transcriptome data gradually regarded as a key information source for predicting complex agronomic traits. However, the existing statistical models (such as linear regression, LASSO) and some machine learning methods (such as random forest, SVM) generally have the following shortcomings: (1) it is difficult to effectively model the complex nonlinear regulation relationship between genes, resulting in limited prediction performance; (2) sensitive to high-dimensional noise, prone to overfitting when the number of genes is much larger than the number of samples; (3) lack of interpretability, making it difficult to identify candidate genes or infer biological mechanisms; (4) unstable prediction performance across species and traits, limiting its promotion in modern breeding. Therefore, there is an urgent need for a phenotype prediction model that can fully exploit local co-expression patterns among gene expressions and achieve dynamic feature selection to improve prediction accuracy and enhance biological interpretability. SUMMARY

[0003] To overcome the above shortcomings, the application aims to provide a GATE intelligent prediction method based on a gated attention mechanism, which can efficiently extract gene expression patterns, enhance key feature selection capabilities, and significantly improve the prediction performance of complex plant traits. At the same time, through the attention and gating signals in the model, gene importance scores can be provided for candidate gene mining and functional research. The application is achieved by the following steps: S1: preprocessing the original transcriptome expression data of the sample to obtain a gene expression vector; inputting the gene expression vector into a multi-channel one-dimensional convolution to extract local co-expression features between genes, obtaining multi-channel convolution features; normalizing and nonlinearly activating the multi-channel convolution features; S2: inputting the activated multi-channel convolution features obtained in step S1 into a channel attention module; in this module, first, a channel-level global description vector is obtained through global average pooling, then the global description vector is transformed through a fully connected layer and an activation function to generate a channel weight vector; the channel weight vector is used to weight the multi-channel convolution features obtained in step S1, and the weighted multi-channel features are outputted; S3: The weighted multi-channel features output from step S2 are simultaneously input into a gating module, which includes a feature path and a gating path. In the feature path, a linear transformation is performed through a fully connected layer to obtain a high-dimensional abstract feature vector. In the gating path, a transformation is performed through a fully connected layer and a sigmoid activation function to generate a dynamic gating weight vector. The high-dimensional abstract feature vector and the dynamic gating weight vector are multiplied element-wise to output the gated features that have been filtered and enhanced. The output gated features are input into a linear extension layer and mapped to the input dimension to obtain extended features. The extended features are added to and fused with the multi-channel features output from step S2 through a residual connection to obtain fused features. The fused features are then subjected to layer normalization. S4: Input the normalized fusion features obtained in step S3 into the regression layer or classification layer for calculation, and output the predicted value of the target trait or the category label.

[0004] Preferably, in step S1, the preprocessing of the original transcriptome expression data of the sample is as follows: the original readings are converted into TPM (transcripts per million), and the TPM value is transformed by natural logarithm ln(TPM+1) to obtain the log expression level, so as to correct the difference in sequencing depth between different samples. Then, genes with a log expression level of zero in more than 80% of the samples are filtered out to reduce noise and improve model efficiency, so as to obtain the gene expression vector.

[0005] Furthermore, in step S1, the Gaussian error linear unit GELU is used as the nonlinear activation function; Preferably, in step S2, the channel attention module adopts a Squeeze-and-Excitation network structure. The process of generating the channel weight vector by transforming through a fully connected layer and a Sigmoid activation function includes: first, a global average pooling layer generates a channel-level global description vector; then, a first fully connected layer reduces the dimension and activates it through ReLU; next, a second fully connected layer increases the dimension; and finally, after activation by Sigmoid, the channel weight vector is output. The model weights the channel features according to the weights, thereby highlighting the feature channels that are highly correlated with the target traits and weakening redundant or noisy signals.

[0006] Preferably, the specific process of transformation through a fully connected layer and a Sigmoid activation function is as follows: the gating path is followed by a Dropout layer after the Sigmoid activation function to generate a sample-specific dynamic gating factor.

[0007] Furthermore, in step S3, the linear expansion layer is a linear transformation layer used to map the hidden space features back to the original gene quantity dimension.

[0008] Preferably, in step S3, the specific process of residual connection is as follows: adding the extended feature to the standardized gene expression vector described in step S1.

[0009] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The efficiency and stability of model construction have been significantly improved. 1) Efficiently capturing local regulatory patterns: A multi-channel one-dimensional convolutional network is used to extract local co-expression features between genes, enabling efficient identification of potential regulatory modules in the high-dimensional transcriptome space. The convolutional layers use the GELU activation function and introduce instance normalization to reduce internal covariate bias, thereby significantly improving the training stability of the model.

[0010] 2) Adaptive enhancement of important features: The Squeeze-and-Excitation structure is introduced. Through global average pooling and channel-wise weighting, the importance of convolutional channels is adaptively adjusted, enabling the model to automatically emphasize feature channels that are more relevant to the target trait.

[0011] 3) Dynamic feature selection mechanism: The attention-weighted features are divided into feature paths and gated paths. The gated paths are activated by Sigmoid to generate dynamic filtering weights. Finally, the key features are automatically selected and amplified through the element-wise product of the two paths, which significantly improves the robustness of the model under noisy data.

[0012] 4) Effective information transmission in deep structures: By combining residual connections and LayerNorm, we ensure stable gradient flow in deep networks, prevent gradient vanishing, and stabilize the activation distribution within the network, thus enabling the model to have better trainability on large-scale transcriptome data.

[0013] 2. The prediction results have good biological interpretability. 1) Gene importance score: The channel attention weights and gating weights output by the model can be multiplied as the importance score at the single gene level, which can be directly used for interpretive analysis and identification of key regulatory genes, and is significantly better than the "black box" characteristics of conventional deep learning models.

[0014] 3. Practicality and scalability 1) Wide applicability across species and multiple traits: The model has simple requirements for input data and can be directly used for the prediction of different plant species and multiple complex traits. It can also be extended to other bioinformatics tasks such as classification.

[0015] 2) Standardized and easy-to-deploy process: This invention has a unified and standardized process from data preprocessing and model training to result interpretation, which can be quickly deployed in actual breeding work and can be seamlessly integrated with existing transcriptome databases or RNA-seq pipelines. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall framework of the present invention; Figure 2 A schematic diagram of the attention mechanism framework of this invention; Figure 3 A schematic diagram of the gating mechanism framework of this invention; Figure 4 A schematic diagram comparing the importance of features generated by various methods. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be further described clearly and completely below with reference to the accompanying drawings. It should be noted that the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0018] This implementation selected data from 388 maize samples and 108 Arabidopsis thaliana samples, sourced from (10.1105 / tpc.19.00332; 10.1038 / s41467-021-25893-w). Please refer to [link / reference needed]. Figures 1-3 As shown, this invention provides a GATE intelligent prediction method based on a gating attention mechanism, and the implementation steps are as follows: S1. First, the transcriptome data of the samples is acquired and preprocessed. The processed data is then input into a multi-channel convolution module. Through one-dimensional convolution across multiple channels, the model can extract local co-expression features between genes. Simultaneously, instance normalization and the GELU (Gaussian Error Linear Unit) activation function are introduced to improve the model's non-linear representation capability and training stability. The GELU function can be expressed as:

[0019] here, Indicates input, Let represent the cumulative distribution function of the standard normal distribution, and erf(·) represent the error function.

[0020] S2. Input the multi-channel convolutional features obtained above into the channel attention module. Generate weight coefficients for each channel through global average pooling and 1×1 convolution, and weight the feature channels based on these weights to highlight channels highly correlated with the target trait. Let the input feature matrix be... Where C represents the number of channels and L represents the sequence length. This module includes three core steps: 1. Global Information Embedding Stage: Global average pooling is performed along the sequence length direction for each channel to extract channel-level global descriptors. This operation effectively integrates the global context of the transcriptome sequence, providing a foundation for modeling inter-channel dependencies and co-regulated gene modules, which can be represented as:

[0021] in, For the first The global average pooling value for each channel. For sequence length, The number of convolution channels. Indicates the first The feature value of each channel at position i.

[0022] 2. Adaptive Recalibration Stage: A bottleneck structure consisting of two fully connected layers is used to perform a nonlinear transformation on the channel descriptors to capture complex inter-channel interactions, which can be represented as:

[0023] in, For channel vector weights, and Here are the weight matrices for dimensionality reduction and dimensionality increase, respectively, and ReLU(·) is the activation function. This represents the sigmoid function, used to generate channel-wise scaling factors. This mechanism models non-mutually exclusive channel dependencies, allowing multiple channels to be emphasized simultaneously, rather than using a one-hot activation method.

[0024] 3. Channel reweighting stage: The learned channel weights are applied to the input features channel by channel to achieve adaptive adjustment, which can be represented as:

[0025] in The output after channel attention weighting. For channel weights, Using the input feature map, the network can dynamically adjust the importance of each channel according to the global context through the above mechanism, thereby enhancing the discriminative ability of the feature representation and improving the generalization performance of the model.

[0026] S3. Subsequently, to enhance the model's ability to select key features, this invention introduces a parallel gating mechanism in the GATE model. Specifically, this structure consists of two parallel fully connected paths: (1) Feature path: linearly transforms the features extracted by the convolutional neural network to obtain a preliminary feature representation. (2) Gating path: generates a gating signal through another fully connected layer to adjust the activation intensity of each channel feature. During forward propagation, the output of the gating path is mapped to the [0,1] interval by the Sigmoid activation function as the channel weight, and further regularized by the Dropout layer. Subsequently, the output of the feature path is multiplied element-wise with the gating coefficient to achieve dynamic feature weighting. This mechanism enables the model to adaptively adjust the importance of each channel according to the input context, thereby improving the sensitivity to key features and enhancing the feature representation capability.

[0027] The Sigmoid activation function can be expressed as:

[0028] in, This indicates input.

[0029] In gene feature importance extraction, this invention fuses dynamic weights generated by gating mechanisms with static weights of feature pathways. Specifically, the average gating coefficient of each feature across samples is calculated and multiplied by its corresponding feature weight to obtain a contribution score. This score more accurately reflects the model's decision-making process. Compared to methods relying solely on static weights, this strategy more fully captures the dynamic adjustment process within the model, thereby improving the interpretability and robustness of feature importance assessment.

[0030] S4. Subsequently, the features output by the channel attention module are mapped back to the dimensional space of the input features through a linear connection layer (expansion layer) to achieve consistency in feature dimensions. Specifically, the expanded and mapped features are residually connected with the channel attention-weighted input features through element-wise addition, thereby completing feature fusion. This residual structure helps to retain the original feature information while introducing the channel attention-modulated expression, and effectively alleviates the gradient vanishing problem that may occur during deep network training. Then, layer normalization is applied to the fused features to stabilize the feature distribution and improve the robustness of model training. Based on the finally obtained feature representation, this invention further performs regression or classification operations to generate prediction results for the target trait.

[0031] In this embodiment, the predictive performance and generalization ability of the model are comprehensively evaluated through five-fold cross-validation, and an early stopping mechanism is introduced to suppress overfitting, thereby improving the stability and reliability of the model in real-world application scenarios.

[0032] Based on the trained model, input the original transcriptome expression data and output the predicted value or category label of the target trait.

[0033] To further verify the advantages of the GATE model in feature interpretation, this embodiment selects multiple highly reliable flowering time genes as a benchmark set (data source: https: / / doi.org / 10.1111 / tpj.13174) and systematically compares the feature importance generated by various methods. Considering the differences in the distribution and dimensions of the importance indices output by different methods, this embodiment converts all importance scores into percentiles to ensure comparability.

[0034] As shown in Table 1, the transcriptome phenotypic prediction model based on the gated attention mechanism proposed in this invention outperforms existing models in predicting multiple traits. Specifically, the GATE model proposed in this invention achieved the highest prediction accuracy across all 10 traits. In core yield traits such as 100 grain weight, ear height, and kernel number per row, GATE's prediction performance is significantly higher than existing methods. In typical developmental traits such as silking time and pollen shedding, GATE achieved accuracies of 0.64 and 0.67, respectively, continuing to maintain its leading position. This indicates that the model can not only identify stable structural expression features but also effectively model time-dependent developmental process signals, remaining reliable in prediction tasks under dynamic regulatory backgrounds. Overall, GATE demonstrates a consistent advantage across a wide range of traits.

[0035] Table 1. Accuracy Comparison of GATE Intelligent Prediction Method and Traditional Methods

[0036] Furthermore, as shown in Figure 4, this model also demonstrates a significant advantage in interpretability. In this embodiment, the gain metrics of LightGBM and XGBoost, the Gini coefficient of Random Forest, the effect value of rrBLUP, and the feature weights generated by the GATE model were compared. The results show that traditional machine learning methods and rrBLUP cannot effectively identify most baseline genes, while the GATE model can more accurately identify flowering time genes related to the target trait, and comprehensively outperforms various existing methods in terms of model interpretability.

[0037] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

Claims

1. A GATE intelligent prediction method based on a gated attention mechanism, characterized in that, Includes the following steps: S1: Preprocess the original transcriptome expression data of the sample to obtain gene expression vectors; input the gene expression vectors into multi-channel one-dimensional convolution to extract local co-expression features between genes, and obtain multi-channel convolution features; The multi-channel convolutional features are normalized and nonlinearly activated. S2: Input the activated multi-channel convolutional features obtained in step S1 into the channel attention module; In this module, a channel-level global description vector is first obtained through global average pooling. Then, the global description vector is transformed through a fully connected layer and an activation function to generate a channel weight vector. The channel weight vector is used to perform channel weighting on the multi-channel convolutional features obtained in step S1, and the weighted multi-channel features are output. S3: Input the weighted multi-channel features output from step S2 into the gating module, which includes a feature path and a gating path; in the feature path, a linear transformation is performed through a fully connected layer to obtain a high-dimensional abstract feature vector; in the gating path, a transformation is performed through a fully connected layer and a Sigmoid activation function to generate a dynamic gating weight vector; the high-dimensional abstract feature vector and the dynamic gating weight vector are multiplied element-wise to output the gated features that have been filtered and enhanced. The output gated features are input into a linear extension layer and mapped to the input dimension to obtain extended features. The extended features are then added to and fused with the output multi-channel features from step S2 through residual connections to obtain fused features. The fused features are then subjected to layer normalization. S4: Input the normalized fusion features obtained in step S3 into the regression layer or classification layer for calculation, and output the predicted value of the target trait or the category label.

2. The method according to claim 1, characterized in that, In step S1, the preprocessing of the original transcriptome expression data of the sample is specifically as follows: the original readings are converted into TPM (transcripts per million), and the TPM value is transformed by the natural logarithm ln(TPM+1) to obtain the log expression level. Then, genes with a log expression level of zero in more than 80% of the samples are filtered out to obtain the gene expression vector.

3. The method according to claim 1, characterized in that, In step S1, the nonlinear activation process uses the Gaussian error linear unit (GELU) as the nonlinear activation function for activation.

4. The method according to claim 1, characterized in that, In step S2, the channel attention module adopts a Squeeze-and-Excitation network structure. The process of generating the channel weight vector by transforming through a fully connected layer and a Sigmoid activation function includes: first, a global average pooling layer generates a channel-level global description vector; then, a first fully connected layer reduces the dimensionality and activates it through ReLU; next, a second fully connected layer increases the dimensionality; and finally, the channel weight vector is output after activation by Sigmoid.

5. The method according to claim 1, characterized in that, In S3, the specific process of transformation through a fully connected layer and a Sigmoid activation function is as follows: the gating path is followed by a Dropout layer after the Sigmoid activation function to generate a sample-specific dynamic gating factor.

6. The method according to claim 1, characterized in that, In step S3, the linear expansion layer is a linear transformation layer used to map the hidden space features back to the original gene quantity dimension.

7. The method according to claim 1, characterized in that, In step S3, the specific process of residual connection is as follows: the extended feature is added to the gene expression vector described in step S1.

Images