SNP site mining method and system based on transformer model ensemble learning

Abstract

The application relates to the technical field of data mining, and discloses a SNP site mining method and system based on a Transformer model ensemble learning, which comprises the following steps: acquiring genetic data containing chromosome SNP sites, and performing screening processing on the genetic data; dividing the screened genetic data into a training set, a verification set and a test set; constructing a basic classifier based on a Transformer model by using the training set and the verification set; analyzing the basic classifier to obtain significant SNP sites; performing ensemble learning on the basic classifier by using the test set, constructing and screening an optimal meta-classifier; and performing classification processing on the significant SNP sites by using the optimal meta-classifier, and screening significant SNP sites related to genetic diseases. By using the Transformer model to construct a basic classifier, analyzing and performing ensemble learning to construct an optimal meta-classifier, and integrating feature information, significant SNP sites related to genetic diseases are finally obtained, and the SNP site mining efficiency and accuracy are improved.

Description

Technical Field

[0001] This invention relates to the field of data mining technology, and more specifically, to a method and system for SNP site mining based on Transformer model ensemble learning. Background Technology

[0002] Research on complex genetic diseases such as Alzheimer's and Parkinson's diseases faces numerous challenges. The pathogenic mechanisms of these diseases are highly complex, and the slow data collection and accumulation process, coupled with typically small sample sizes, slows research progress. Furthermore, these diseases often involve multiple genetic variations and patterns, meaning that different individuals may exhibit similar disease phenotypes due to different mutations. This genetic heterogeneity makes identifying potential disease-causing genes more difficult. Traditional genome-wide association studies (GWAS) primarily focus on the linear effects of individual SNPs (Single Nucleotide Polymorphisms), typically neglecting the complex nonlinear interactions between SNP sites.

[0003] Existing SNP mining techniques typically rely on linear statistical models to identify individual SNP loci associated with diseases. However, this approach has significant limitations when dealing with complex genetic diseases. Since complex diseases are often caused by nonlinear interactions between multiple genes, traditional linear models often fail to capture the nonlinear associations between SNP loci and phenotypes in small datasets. This makes it difficult to accurately capture both linear and nonlinear associations between SNP loci and phenotypes using small datasets, thus hindering the discovery of relevant SNP loci. Consequently, the accuracy and efficiency in detecting disease-related SNP loci are low. Summary of the Invention

[0004] To overcome the shortcomings of low accuracy and efficiency in existing SNP mining techniques, this invention proposes the following technical solution:

[0005] Firstly, this invention proposes a method for SNP site mining based on Transformer model ensemble learning, comprising:

[0006] Obtain genetic data containing chromosomal SNP loci and perform screening processing on the genetic data;

[0007] The filtered genetic data is divided into training, validation, and test sets.

[0008] Using the training and validation sets, construct a basic classifier based on the Transformer model;

[0009] The basic classifier was analyzed to obtain significant SNP sites;

[0010] The base classifiers are ensemble-learned using the test set to construct and select the optimal meta-classifier;

[0011] Significant SNP sites were classified using an optimal meta-classifier to screen out significant SNP sites associated with genetic diseases.

[0012] As a preferred technical solution, the genetic data is screened and processed, including:

[0013] The genetic data were preprocessed sequentially, including unifying coordinates, deleting outliers, and imputation repair.

[0014] Calculate the p-value of the SNP loci on each chromosome in the preprocessed genetic data.

[0015] For each chromosome, the top 128 SNP loci with the smallest p-values ​​are extracted as the selected genetic data.

[0016] As a preferred technical solution, after obtaining significant SNP sites, the method further includes:

[0017] Genes associated with significant SNP sites were pooled to obtain gene sets.

[0018] Enrichment analysis of gene sets was performed to construct biological pathways related to genetic diseases.

[0019] As a preferred technical solution, a basic classifier based on the Transformer model is constructed using the training set and validation set, including:

[0020] Calculate the feature representations of genetic data in the training and validation sets.

[0021] Based on the absolute location information of SNP sites on the chromosome, the feature representation is positionally embedded to obtain a feature representation with SNP site location information.

[0022] Using the Transformer model as a deep learning architecture, 22 basic classifiers were built, each corresponding to one of the 22 autosomal data.

[0023] The base classifier is trained using feature representations with SNP location information until the preset conditions are met, resulting in a well-trained base classifier.

[0024] As a preferred technical solution, the feature representations of genetic data in the training and validation sets are calculated according to the following formula:

[0025]

[0026]

[0027] Where W1 and b1 are the weight matrix and bias vector of the linear layer, respectively. Indicates a hot coding operation. h The eigenvectors are the result of a linear transformation. x type A vector representing the mutation type of an SNP site. x site This vector represents the number of mutated SNP sites. This indicates a merge process. X These are the feature vectors after merging.

[0028] As a preferred technical solution, the feature representation is positionally embedded based on the absolute position information of the SNP site on the chromosome, and its expression is as follows:

[0029]

[0030]

[0031] in, i Indicates the position index. j The dimension index of the embedded vector, pos_idx[ i [] represents the index of the i-th SNP site on the chromosome, and embedding_dim is the dimension of the embedding vector.

[0032] As a preferred technical solution, the basic classifier is analyzed to obtain significant SNP sites, including:

[0033] The self-attention matrix of the base classifier at layer d and head h is calculated as follows:

[0034]

[0035] Where L is the number of SNP sites in the input genetic data. This indicates the CLS tag.

[0036] The final self-attention matrix of the base classifier is obtained by averaging all self-attention matrices sequentially along the head dimension and layer dimension. Its expression is as follows:

[0037]

[0038]

[0039] Where H is the number of heads, representing the number of paths that the base classifier processes in parallel when processing data, and D is the number of layers, representing the depth of the base classifier.

[0040] From the final self-attention matrix The first row extracts the association score between the CLS marker and each SNP site. And calculate the association score based on the normal distribution. p-value Its expression is as follows:

[0041]

[0042]

[0043] ， ~N (0,1)

[0044] in, This represents the row vectors associated with the CLS label extracted from the self-attention matrix, with each element... Indicates the CLS tag and the first l Association score of each SNP site.

[0045] Given a significance level α, if <α, then it is considered that the first l The SNP sites are significant SNP sites.

[0046] As a preferred technical solution, the base classifier is ensemble-learned using a test set to construct and select the optimal meta-classifier, including:

[0047] Calculate the Z-score of each base classifier on the test set.

[0048] Set several decreasing probability values ​​and use the cumulative distribution function to calculate the Z-score threshold corresponding to each probability value.

[0049] The basic classifiers are filtered sequentially using each Z-score threshold, generating several sets of basic classifiers, including a set of basic classifiers containing all basic classifiers.

[0050] Construct a corresponding meta-classifier using each set of basic classifiers, and use the meta-classifier containing all basic classifiers as the baseline meta-classifier.

[0051] Calculate the AUC score of each meta-classifier, and select the meta-classifier that contains the fewest base classifiers and whose AUC score is closest to the baseline meta-classifier as the optimal meta-classifier.

[0052] As a preferred technical solution, a corresponding meta-classifier is constructed using each set of basic classifiers, including:

[0053] The meta-classifier extracts the feature vectors output by each base classifier from the base classifier set and fuses these features to obtain an ensemble feature vector. This ensemble feature vector is then activated sequentially through a fully connected layer and a softmax activation function to obtain the classification result.

[0054] Secondly, this invention also proposes an SNP site mining system based on Transformer model ensemble learning, applicable to the SNP site mining method based on Transformer model ensemble learning as described in any of the schemes of the first aspect, comprising:

[0055] The acquisition module acquires genetic data containing chromosomal SNP loci and performs filtering processing on the genetic data;

[0056] The partitioning module is used to divide genetic data into training, validation, and test sets.

[0057] The building blocks are used to construct a base classifier based on the Transformer model using the training and validation sets;

[0058] The parsing module is used to parse the basic classifier and obtain significant SNP sites;

[0059] The ensemble learning module is used to perform ensemble learning on the base classifier using the test set, and to build and select the optimal meta-classifier.

[0060] The classification module is used to classify significant SNP sites using the optimal meta-classifier and screen out significant SNP sites associated with genetic diseases.

[0061] The beneficial effects of the present invention include at least the following:

[0062] This invention, through screening and rationally dividing genetic data, fully considers the biological properties and characteristics of the data and maximizes the utilization of key data information, laying the foundation for subsequent data mining tasks. The basic classifier built based on the Transformer model utilizes the location information of SNP loci, reducing the required training sample size. Significant SNP loci are obtained through analysis of the basic classifier, and ensemble learning is performed using the test set to construct an optimal meta-classifier, fully integrating the feature information of each classifier. Finally, the optimal meta-classifier is used to classify significant SNP loci, screening out significant SNP loci associated with genetic diseases. This series of steps not only reduces the computational complexity and storage requirements of the algorithm, improving mining efficiency and accuracy, but also uncovers more relevant genetic loci, providing strong support for genetic disease research. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating the SNP site mining method based on Transformer model ensemble learning provided in Example 1.

[0064] Figure 2 This is a framework diagram of the SNP site mining method based on Transformer model ensemble learning provided in Example 2.

[0065] Figure 3 This is an architecture diagram of the SNP site mining system based on Transformer model ensemble learning provided in Example 3. Detailed Implementation

[0066] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred technical solutions. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred technical solutions are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0067] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0068] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0069] Example 1

[0070] This embodiment proposes a SNP site mining method based on Transformer model ensemble learning, such as... Figure 1 As shown, Figure 1 This is a flowchart illustrating a SNP site mining method based on Transformer model ensemble learning provided in this embodiment. The method includes the following steps:

[0071] S1: Obtain genetic data containing chromosomal SNP loci and perform screening processing on the genetic data.

[0072] S2: Divide the selected genetic data into training, validation and test sets.

[0073] S3: Using the training and validation sets, construct a basic classifier based on the Transformer model.

[0074] S4: Analyze the basic classifier to obtain significant SNP sites.

[0075] S5: Use the test set to perform ensemble learning on the base classifiers, and build and select the optimal meta-classifier.

[0076] S6: Use the optimal meta-classifier to classify significant SNP sites and screen out significant SNP sites associated with genetic diseases.

[0077] Understandably, this embodiment, through screening and rationally dividing genetic data, fully considers the biological properties and characteristics of the data and maximizes the utilization of key data information, laying the foundation for subsequent data mining tasks. The basic classifier built based on the Transformer model utilizes the location information of SNP sites, reducing the training sample size requirement. Significant SNP sites are obtained through analysis of the basic classifier, and ensemble learning is performed using the test set to construct the optimal meta-classifier, fully integrating the feature information of each classifier. Finally, the optimal meta-classifier is used to classify significant SNP sites, screening out significant SNP sites related to genetic diseases. This series of steps not only reduces the computational complexity and storage requirements of the algorithm, improving mining efficiency and accuracy, but also uncovers more relevant genetic sites, providing strong support for genetic disease research.

[0078] Example 2

[0079] like Figure 2 As shown, this embodiment improves upon the SNP site mining method based on Transformer model ensemble learning proposed in Embodiment 1, as follows: Figure 2 As shown, Figure 2 This is a diagram illustrating the implementation framework of the SNP site mining method based on Transformer model ensemble learning provided in this embodiment.

[0080] This embodiment downloads the ADNI1 and ADNI2GO datasets from the ADNI database, and the Project107 dataset (hereinafter referred to as the PPMI dataset) from the PPMI database. Genotyping data samples are obtained from the ADNI1, ADNI2GO, and PPMI datasets. The ADNI1, 2, and GO datasets contain three groups: Alzheimer's disease patients (AD), mild cognitive impairment (MCI), and normal controls (HC). The PPMI dataset contains three groups: Parkinson's disease patients (PD), scans without evidence of dopaminergic deficiency (SWEDD), and normal controls (HC). SWEDD is a term used in Parkinson's disease research, referring to patients exhibiting clinical symptoms of Parkinson's disease but without evidence of dopaminergic system deficiency on brain imaging scans. The initial data of the ADNI1 and ADNI2GO datasets are shown in Table 1.

[0081] Table 1 ADNI Dataset Information

[0082]

[0083] Subsequently, the ADNI1 and ADNI2GO datasets were merged. Using LiftOver software, the coordinates of ADNI1 were transferred from Hg18 to Hg19, resulting in ADNI1 containing 472,304 SNP loci. Site correction, inversion, and quality control were performed on ADNI1 and ADNI2GO using the hg19 Human Genome Project reference gene as a template, and the intersection of their SNP loci was taken. Due to duplicate participants in ADNI Phase 1 and Phase 2 and GO projects, identical samples were removed, resulting in a hybrid ADNI dataset (hereinafter referred to as the ADNI dataset). Therefore, two datasets were obtained: the ADNI dataset and the PPMI dataset. The data characteristics of the two datasets are shown in Table 2.

[0084] Table 2 Information on ADNI and PPMI datasets

[0085]

[0086] In this embodiment, the genetic data is screened, including:

[0087] The genetic data were preprocessed sequentially, including unifying coordinates, deleting outliers, and imputation repair.

[0088] Calculate the p-value of the SNP loci on each chromosome in the preprocessed genetic data.

[0089] For each chromosome, the top 128 SNP loci with the smallest p-values ​​are extracted as the selected genetic data.

[0090] In the specific implementation process, strict quality control and imputation repair were performed on the genetic data. Plink software was used for quality control operations, with the following parameters: deletion of SNPs and individuals with a deletion rate exceeding 20%, setting the minimum allele frequency to 0.05, a hard violation threshold of Hardy-Weinberg equilibrium to 1e-6, independence screening to output independent SNPs, and deletion of individuals with the lowest call rate among correlation pairs with pihat>0.2. Then, population stratification was performed to eliminate population differences. Finally, imputation repair was performed online using the Michigan imputation server website. When performing imputation on the Michigan online imputation website, the Minimac4 imputation tool was used. The parameters were set as follows: data type unphased, reference genome version hg19, reference panel apps@1000g-phase-3-v5 (hg19), population group European (EUR), phasing tool eagle, mode imputation, and Rsq filter value set to 0.3.

[0091] The ADNI dataset was processed as follows: Since the ADNI data originates from Europe, non-European samples were removed. First, the human reference genome at hg19 coordinates and the ADNI dataset underwent quality control. Next, they were merged. Because the previous two datasets, ADNI1 and ADNI2GO, were merged according to this standard, merging the reference genome with the ADNI dataset did not result in any loss of loci. Then, PCA population distribution was performed, generating a multidimensional scaling analysis plot and displaying the distribution across the first 10 dimensions. Finally, 940 samples closest to the European samples were selected. After imputation and further quality control, 5,361,007 variable loci and 940 samples were obtained.

[0092] The PPMI dataset was processed as follows: Since the PPMI data originates from Europe, non-European samples were removed. First, the dataset was converted from hg18 to hg19 coordinates using LiftOver software. Second, the hg19 coordinate human reference genome and PPMI dataset were quality controlled. They were then merged, and PCA population distribution was performed to generate a multidimensional scaling analysis plot, displaying the distribution across the first 10 dimensions. Finally, 461 samples closest to the European samples were selected. After interpolation and further quality control, 1,192,960 variable loci and 461 samples were obtained.

[0093] The two datasets were then processed. The MCI and AD samples were merged into one class, called the anomalous sample (Disease), and the SWEDD and PD samples were also merged into one class (Disease) before proceeding to downstream tasks. The preprocessed sample data of the resulting datasets are shown in Table 3.

[0094] Table 3. Sample data after preprocessing of the ADNI and PPMI datasets.

[0095]

[0096] During the initial screening of genetic data, the plink software was used to perform association analysis on the preprocessed data. The logistic model was used to perform logistic regression analysis to calculate the p-value of each variable site, and the top 128 variable sites with the smallest p-values ​​on each chromosome were extracted as data for subsequent tasks.

[0097] In this embodiment, a stratified random sampling strategy was used to partition the dataset, dividing the entire dataset into a training set, validation set, and test set in a 3:1:1 ratio. During the partitioning process, the proportion of disease samples to normal samples was ensured to be consistent across all datasets. Furthermore, four-fold cross-validation was performed on the training and validation sets, ultimately generating four sets of data. The test set is an independent dataset that does not participate in any training and is only used for final model performance evaluation. In this embodiment, 187 samples were partitioned from the ADNI dataset (940 samples in total) as an independent test set, which does not participate in any training and is only used for final model performance evaluation. The remaining 753 samples underwent four-fold cross-validation to obtain the original training set and original validation set for each fold. Therefore, each fold contains 563 samples in the original training set, 187 samples in the original validation set, and 187 samples in the test set. The sample distribution of the original training set and original validation set for each fold is shown in Table 4, and the sample distribution of the original test set is shown in Table 5.

[0098] Table 4. Sample distribution of the original training set and original validation set

[0099]

[0100] Table 5. Distribution of Original Test Set Samples

[0101]

[0102] To improve the model's ability to represent and understand data and to find more disease-related variable loci, a basic classifier is constructed using chromosome partitioning. The basic classifier is then trained, and the learning results are interpreted using a self-attention matrix to identify disease-related variable loci. This embodiment utilizes sine and cosine positional encoding techniques and specific data type representation methods to represent the data, using the Transformer as the basic architecture of the base model to construct a scheme for finding relevant variable loci.

[0103] In this embodiment, a basic classifier based on the Transformer model is constructed using the training set and validation set, including:

[0104] Calculate the feature representations of genetic data in the training and validation sets:

[0105]

[0106]

[0107] Where W1 and b1 are the weight matrix and bias vector of the linear layer, respectively. Indicates a hot coding operation. h The eigenvectors are the result of a linear transformation. x type A vector representing the mutation type of an SNP site. x site This vector represents the number of mutated SNP sites. This indicates a merge process. X The merged feature vectors represent the merging of these vectors into a single event, which has 40 possible outcomes.

[0108] Given that the classification model architecture consists of 22 basic classifiers, each corresponding to a chromosome, the positional information of variable sites on the chromosome can be used as positional embedding features. This embodiment utilizes the absolute positional information of SNP sites on the chromosome and performs positional embedding based on sine and cosine functions. To facilitate the encoding of positional information in a deep learning neural network architecture, this embodiment employs a method of embedding positions in a sequence. Based on the absolute positional information of the SNP sites on the chromosome, positional embedding is performed on the feature representation to obtain a feature representation with SNP site positional information, providing positional information for each position in the input sequence. The expression is as follows:

[0109]

[0110]

[0111] in, i Indicates the position index. j The dimension index of the embedded vector, pos_idx[ i [] represents the index of the i-th SNP site on the chromosome, and embedding_dim is the dimension of the embedding vector.

[0112] In this embodiment, a chromosome-based base classifier is constructed using the Transformer deep learning architecture. A total of 22 base classifiers are established, each corresponding to one of the 22 autosomal chromosomes. Appropriate hyperparameters are selected for training based on the training curve to ensure that all base classifiers converge. The specific model architecture of the base classifier in this embodiment is described below:

[0113] Each base classifier corresponds to one chromosome, ensuring that each base classifier processes data from a single chromosome (therefore, a total of 22 base classifiers correspond to 22 autosomes). The input includes the genotype of each SNP locus (n=128, the first 128 SNP loci with the smallest chromosome p-value) and its position on the corresponding chromosome. These two types of information are processed separately. For the genotype data at each locus, the input dimension is 40, extended to 512 dimensions through word embeddings. The position information of each SNP locus on the corresponding chromosome is further processed using position embedding methods, mapping it to a 512-dimensional space. Then, the processed data from both information sources are combined additively and concatenated with classification labels used for classification. Next, the data is processed through a self-attention mechanism with an 8-layer structure consisting of 12 heads. Finally, a 512x2 fully connected layer is used to generate binary classification probabilities (healthy individual or patient).

[0114] The base classifier is trained using feature representations incorporating SNP location information until preset conditions are met, resulting in a well-trained base classifier. The cross-entropy loss function is used, as shown in the following formula:

[0115]

[0116] The model was trained and selected using training and validation sets. Each column represents a chromosome, and each row represents the AUC score of the corresponding validation set in four-fold cross-validation. The base classifiers had the same structure and an initial learning rate of 5e-5. The learning rate was adjusted using the CosineAnnealingLR learning rate decay strategy over 300 training epochs. The Adam optimizer was used. The final results are shown in Tables 6 and 7.

[0117] Table 6. Results of the base classifier on the ADNI validation set.

[0118] Table 7 Results of the base classifier on the PPMI validation set.

[0119] In this embodiment, the basic classifier is analyzed to obtain significant SNP sites, including:

[0120] The self-attention matrix of the base classifier at layer d and head h is calculated as follows:

[0121]

[0122] Where L is the number of SNP sites in the input genetic data. This indicates the CLS tag.

[0123] The final self-attention matrix of the base classifier is obtained by averaging all self-attention matrices sequentially along the head dimension and layer dimension. Its expression is as follows:

[0124]

[0125]

[0126] Where H is the number of heads, representing the number of paths that the base classifier processes in parallel when processing data, and D is the number of layers, representing the depth of the base classifier.

[0127] From the final self-attention matrix The first row extracts the association score between the CLS marker and each SNP site. And calculate the association score based on the normal distribution. p-value Its expression is as follows:

[0128]

[0129]

[0130] ， ~N (0,1)

[0131] in, This represents the row vectors associated with the CLS label extracted from the self-attention matrix, with each element... Indicates the CLS tag and the first l Association score of each SNP site.

[0132] Given a significance level α, if <α, then it is considered that the first l Each SNP site is considered a significant SNP site. α is set to 0.0001 or 0.05.

[0133] In practice, α is set to 0.0001 to identify highly significant SNP sites, and α is set to 0.05 to identify relatively less significant SNP sites.

[0134] for Significant SNP sites with a value <0.0001 are considered highly significant and can be directly used for reporting as the main research focus.

[0135] For 0.0001 ≤ Significant SNP sites <0.05, although these sites are not as... Significant SNP sites with a value <0.0001 are not significant, but are still statistically significant and are mainly used for subsequent pathway analysis to reveal broader biological processes.

[0136] In this embodiment, Significant SNP sites with a value <0.0001 are listed, along with their corresponding associated genes retrieved from the dbSNP database. Genes reported to be associated with the disease are highlighted in bold. Results are shown in Tables 8 and 9.

[0137] Table 8 shows the results of identifying significant SNP sites on each chromosome (ADNI dataset).

[0138]

[0139] Table 9 shows the results of identifying significant SNP sites on each chromosome (PPMI dataset).

[0140]

[0141] In this embodiment, the base classifier is ensemble-learned using a test set to construct and select the optimal meta-classifier, including:

[0142] Calculate the Z-score of each base classifier on the test set.

[0143] Set several decreasing probability values ​​and use the cumulative distribution function to calculate the Z-score threshold corresponding to each probability value.

[0144] The basic classifiers are filtered sequentially using each Z-score threshold, generating several sets of basic classifiers, including a set of basic classifiers containing all basic classifiers.

[0145] Calculate the AUC score of each meta-classifier, and select the meta-classifier that contains the fewest base classifiers and whose AUC score is closest to the baseline meta-classifier as the optimal meta-classifier.

[0146] In the specific implementation process, the Z-score method is used to distinguish the basic classifiers into different tiers, divided into six tiers based on one-sided test intervals. Then, the basic classifiers included in these six tiers are integrated for learning, constructing six meta-models. These meta-models are trained using training and validation sets, resulting in six meta-models. Finally, the meta-model that performs better with less data on the test set is selected, and its corresponding chromosome is used as a key chromosome for subsequent biological pathway mining.

[0147] Then, the method of selecting a meta-model that performs well with less data on the test set is novel. The AUC results of each meta-model on the test set are compared, and the best model that represents all data with the least amount of data is selected. Specifically, the best model should meet two conditions: Condition 1, the ratio of its AUC score to the AUC of the meta-model using all chromosomes should be higher than 98%. Condition 2, based on meeting Condition 1, the meta-model with the least amount of data is selected.

[0148] The meta-training strategy is as follows: Each meta-classifier is trained for 50 epochs using the Adam optimizer with an initial learning rate of 5e-5 and a CosineAnnealingLR learning rate decay strategy. The p-value is obtained by performing a one-tailed hypothesis test on the AUC scores of the base classifiers. After training, the test set results are shown in Tables 10 and 11. The upper table in Tables 10 and 11 represents the AUC score of the meta-model for each chromosome set on the test set, and the lower table represents the chromosome numbers contained in each chromosome set.

[0149] Table 10 shows the performance results of the 10-gram model on the test set (ADNI dataset).

[0150]

[0151] Table 11 shows the performance results of the 11-meta-model on the test set (PPMI dataset).

[0152]

[0153] Through performance testing of the meta-model, the key chromosomes in the two datasets were finally identified. For the ADNI dataset, the set of chromosomes with P>=0.5 satisfied the selection criteria. For the PPMI dataset, the set of chromosomes with P>=0.1 satisfied the selection criteria.

[0154] This embodiment tests the performance of ensemble learning and non-ensemble learning methods on this classification task. To verify the effectiveness of ensemble learning in improving model performance, ensemble learning is set aside, excluding the concepts of meta-model and base classifier. Instead, the same architecture as the base classifier is used directly to train and evaluate a total of 2816 extraction locations (2816 = 22 x 128), which is the non-ensemble learning method. Here, the selected location information embedding method is trainable location embedding. The training set, validation set, and test set remain consistent with the previous settings.

[0155] Due to the large number of computational locations, the batch size for the samples was set to 2 to ensure that the model could converge considering the available GPU memory limitations. The final results are shown in Table 12. Clearly, the model using ensemble learning outperforms the model without ensemble learning.

[0156] Table 12 shows the test results of using ensemble learning and non-ensemble learning methods on the ADNI dataset.

[0157]

[0158] In constructing the corresponding meta-classifier using each set of base classifiers, the weights of each base classifier are fixed, and the feature output of the penultimate layer of each base classifier is extracted. This feature output is then used as the input to the meta-model. Each base classifier is responsible for one autosome, and CLStoken is a summary of the data for that autosome from each base classifier. Therefore, CLStoken is helpful in distinguishing between normal and diseased samples. Integrating the summaries obtained from each base classifier is equivalent to combining the classification-useful information from all 22 autosomes. This integrated information is then used as the input to the meta-model for ensemble learning.

[0159] This embodiment uses a simple averaging fusion strategy for the integrated learning meta-model component. The specific steps are as follows:

[0160] (1) Weight allocation: Assume there are M base classifiers. Define M weight parameters W. m Where m = 1, 2, ..., M. Each base classifier has the same weight parameter, with a value of 1 / M, meaning that for any base classifier m, its weight parameter W... m =1 / M.

[0161] (2) Feature fusion: The second-to-last layer feature mapping of the m-th base classifier is represented as vector h. m h m With the corresponding weight W m Multiplication: Z m =W m *h mFor any m, concatenate the M weighted feature vectors to obtain the integrated feature: Z = [Z1, Z2, ..., Zn]. M ].

[0162] (3) Final classification: Input Z into a fully connected layer and pass it through the softmax activation function to obtain the binary classification probability, which is expressed as:

[0163]

[0164] Where Wz and β are learnable weight matrices and biases, respectively. P is the output vector, where [P1, P2] represents the classification probability.

[0165] In this embodiment, after obtaining significant SNP sites, the method further includes:

[0166] Genes associated with significant SNP sites were pooled to obtain gene sets.

[0167] Enrichment analysis of gene sets was performed to construct biological pathways related to genetic diseases.

[0168] In the specific implementation process, the R language package clusterProfiler was used to calculate the enrichment p-value of each pathway term in the gene set G contained in the key chromosome based on hypergeometric distribution, and significant pathways (p<0.05) were selected. The specific formula for hypergeometric distribution calculation is as follows:

[0169]

[0170] Where N is the number of background genes, M is the number of pathway genes, n is the number of test genes, and m is the number of test genes included in the pathway. The significant pathway results are shown in Table 12; pathways reported to be disease-related are highlighted in bold.

[0171] Table 12 shows the significant biological pathways obtained from the two datasets.

[0172]

[0173] Example 3

[0174] like Figure 3 As shown, this embodiment proposes an SNP site mining system based on Transformer model ensemble learning, which is applied to the SNP site mining method based on Transformer model ensemble learning as described in the above embodiment. It includes: an acquisition module 100, a partitioning module 200, a construction module 300, a parsing module 400, an ensemble learning module 500, and a classification module 600.

[0175] The system comprises the following modules: Acquisition module 100 acquires genetic data containing chromosomal SNP loci and performs filtering on the data; Segmentation module 200 divides the genetic data into training, validation, and test sets; Construction module 300 constructs a basic classifier based on the Transformer model using the training and validation sets; Parsing module 400 parses the basic classifier to obtain significant SNP loci; Ensemble learning module 500 performs ensemble learning on the basic classifier using the test set to construct and select the optimal meta-classifier; and Classification module 600 classifies the significant SNP loci using the optimal meta-classifier to select significant SNP loci associated with genetic diseases.

[0176] It should be noted that the foregoing explanation of the SNP site mining method based on Transformer model ensemble learning also applies to the SNP site mining system based on Transformer model ensemble learning in this embodiment, and will not be repeated here.

[0177] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0178] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0179] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0180] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0181] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0182] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A SNP site mining method based on a Transformer model ensemble learning, characterized in that, include: Obtain genetic data containing chromosomal SNP loci and perform screening processing on the genetic data; The filtered genetic data is divided into training, validation, and test sets. Using the training and validation sets, a basic classifier based on the Transformer model is constructed, including: Calculate the feature representations of genetic data in the training and validation sets; Based on the absolute position information of SNP sites on the chromosome, the feature representation is positionally embedded to obtain a feature representation with SNP site position information; Using the Transformer model as a deep learning architecture, 22 basic classifiers were built, each corresponding to one of the 22 autosomal data. The base classifier is trained using feature representations with SNP location information until the preset conditions are met, resulting in a well-trained base classifier. The basic classifier was analyzed to obtain significant SNP sites; The base classifiers are ensemble-learned using the test set to construct and select the optimal meta-classifier; Significant SNP sites were classified using an optimal meta-classifier to screen out significant SNP sites associated with genetic diseases.

2. The SNP site mining method based on Transformer model ensemble learning according to claim 1, characterized in that, The genetic data is screened, including: The genetic data were preprocessed sequentially, including unifying coordinates, removing outliers, and imputation repair. Calculate the p-value of the SNP locus on each chromosome in the preprocessed genetic data; For each chromosome, the top 128 SNP loci with the smallest p-values ​​are extracted as the selected genetic data.

3. The SNP site mining method based on Transformer model ensemble learning according to claim 1, characterized in that, After obtaining significant SNP sites, the method further includes: Genes associated with significant SNP sites were pooled to obtain gene sets. Enrichment analysis of gene sets was performed to construct biological pathways related to genetic diseases.

4. The SNP site mining method based on Transformer model ensemble learning according to claim 1, characterized in that, The feature representations of the genetic data in the training and validation sets are calculated according to the following formula: Where W1 and b1 are the weight matrix and bias vector of the linear layer, respectively. Indicates a hot coding operation. h The eigenvectors are the result of a linear transformation. x type A vector representing the mutation type of an SNP site. x site This vector represents the number of mutated SNP sites. This indicates a merge process. X These are the feature vectors after merging.

5. The SNP site mining method based on Transformer model ensemble learning according to claim 1, characterized in that, Based on the absolute location information of the SNP sites on the chromosome, the feature representation is embedded positionally, and its expression is as follows: in, i Indicates the position index. j The dimension index of the embedded vector, pos_idx[ i [] represents the index of the i-th SNP site on the chromosome, and embedding_dim is the dimension of the embedding vector.

6. The SNP site mining method based on Transformer model ensemble learning according to claim 1, characterized in that, The base classifier was analyzed to obtain significant SNP sites, including: The self-attention matrix of the base classifier at layer d and head h is calculated as follows: Where L is the number of SNP sites in the input genetic data. Indicates the CLS tag; The final self-attention matrix of the base classifier is obtained by averaging all self-attention matrices sequentially along the head dimension and layer dimension. Its expression is as follows: Where H is the number of heads, representing the number of paths that the base classifier processes in parallel when processing data, and D is the number of layers, representing the depth of the base classifier; From the final self-attention matrix The first row extracts the association score between the CLS marker and each SNP site. And calculate the association score based on the normal distribution. p-value Its expression is as follows: ， ~ N (0, 1) in, This represents the row vectors associated with the CLS markers extracted from the self-attention matrix, with each element... Indicates the CLS tag and the first l Association score of each SNP site; Given a significance level α, if If < α, then the first l The SNP sites are significant SNP sites.

7. The SNP site mining method based on Transformer model ensemble learning according to claim 1, characterized in that, Ensemble learning is performed on the base classifiers using the test set to construct and select the optimal meta-classifier, including: Calculate the Z-score of each base classifier on the test set; Set several decreasing probability values ​​and use the cumulative distribution function to calculate the Z-score threshold corresponding to each probability value; The basic classifiers are filtered sequentially using each Z-score threshold, generating several sets of basic classifiers, including a set of basic classifiers containing all basic classifiers. Construct a corresponding meta-classifier using each set of basic classifiers, and use the meta-classifier containing all basic classifiers as the baseline meta-classifier; Calculate the AUC score of each meta-classifier, and select the meta-classifier that contains the fewest base classifiers and whose AUC score is closest to the baseline meta-classifier as the optimal meta-classifier.

8. The SNP site mining method based on Transformer model ensemble learning according to claim 7, characterized in that, Construct the corresponding meta-classifier using each set of base classifiers, including: The meta-classifier extracts the feature vectors output by each basic classifier in the set of basic classifiers and performs feature fusion to obtain an integrated feature vector; the integrated feature vector is then activated sequentially through a fully connected layer and a softmax activation function to obtain the classification result.

9. A SNP site mining system based on Transformer model ensemble learning, applied to the SNP site mining method based on Transformer model ensemble learning as described in any one of claims 1 to 8, characterized in that, include: The acquisition module acquires genetic data containing chromosomal SNP loci and performs filtering processing on the genetic data; The partitioning module is used to divide genetic data into training, validation, and test sets. The building blocks are used to construct a base classifier based on the Transformer model using the training and validation sets; The parsing module is used to parse the basic classifier and obtain significant SNP sites; The ensemble learning module is used to perform ensemble learning on the base classifier using the test set, and to build and select the optimal meta-classifier. The classification module is used to classify significant SNP sites using the optimal meta-classifier and screen out significant SNP sites associated with genetic diseases.

Images