SNP (Single Nucleotide Polymorphism) site mining method and system based on Transform model ensemble learning

Abstract

The invention relates to the technical field of data mining, and discloses an SNP site mining method and system based on Transform model ensemble learning, and the method comprises the steps: obtaining genetic data containing chromosome SNP sites, and carrying out the screening processing of the genetic data; dividing the screened genetic data into a training set, a verification set and a test set; using the training set and the verification set to construct a basic classifier based on a Transform model; analyzing the basic classifier to obtain a remarkable SNP (Single Nucleotide Polymorphism) site; carrying out integrated learning on the basic classifier by utilizing the test set, and constructing and screening out an optimal element classifier; and carrying out classification processing on the significant SNP sites by utilizing the optimal meta classifier, and screening out the significant SNP sites related to the genetic diseases. According to the method, the basic classifier is constructed by using the Transform model, the optimal element classifier is constructed through analysis and ensemble learning, and the feature information is integrated, so that the significant SNP loci related to the genetic diseases are finally obtained, and the mining efficiency and accuracy of the SNP loci are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data mining, and more specifically, to a SNP site mining method and system based on Transformer model ensemble learning. Background Art

[0002] The study of complex genetic diseases such as Alzheimer's disease and Parkinson's disease faces many challenges. The pathogenic mechanisms of these diseases are very complex, and the sample size is usually small due to the slow data collection and accumulation process, which makes the research progress slow. In addition, these diseases often involve multiple genetic variants and inheritance patterns, and different individuals may present similar disease phenotypes due to different mutations. Genetic heterogeneity makes it more difficult to identify potential pathogenic genes. Traditional genome-wide association studies (GWAS) mainly focus on the linear effects of single SNP (Single Nucleotide Polymorphism) sites, and usually ignore the complex nonlinear interactions between SNP sites.

[0003] Existing SNP mining techniques usually rely on linear statistical models and aim to identify single SNP sites associated with diseases. However, this approach has significant defects when dealing with complex genetic diseases. Since complex diseases are mostly caused by nonlinear interactions between multiple genes, traditional linear models often cannot capture the nonlinear associations between SNP sites and phenotypes in small data sets, and cannot accurately capture the linear and nonlinear associations between SNP sites and phenotypes through small data sets to discover related SNP sites, which leads to low accuracy and efficiency in detecting SNP sites associated with diseases. Summary of the invention

[0004] In order to overcome the defects of low accuracy and efficiency in the existing SNP mining technology, the present invention proposes the following technical solutions:

[0005] In the first aspect, the present invention proposes a SNP site mining method based on Transformer model ensemble learning, comprising:

[0006] Obtain genetic data containing chromosome SNP sites, and screen and process the genetic data;

[0007] Divide the screened genetic data into training set, validation set and test set;

[0008] Use the training set and validation set to build a basic classifier based on the Transformer model;

[0009] Analyze the basic classifier to obtain significant SNP sites;

[0010] Use the test set to perform ensemble learning on the basic classifiers, build and select the optimal meta-classifier;

[0011] The optimal meta-classifier is used to classify the significant SNP sites and screen out the significant SNP sites related to genetic diseases.

[0012] As a preferred technical solution, the genetic data is screened and processed, including:

[0013] The genetic data were preprocessed in sequence, including unifying coordinates, deleting outliers, and interpolating and repairing.

[0014] Calculate the p-value of the SNP site on each chromosome in the preprocessed genetic data.

[0015] For each chromosome, the first 128 SNP sites with the smallest p-value in each chromosome were extracted as the filtered genetic data.

[0016] As a preferred technical solution, after obtaining the significant SNP site, the method further includes:

[0017] Genes associated with significant SNP sites were pooled to obtain a gene set.

[0018] Perform enrichment analysis on gene sets and construct biological pathways related to genetic diseases.

[0019] As a preferred technical solution, a basic classifier based on the Transformer model is constructed using the training set and the validation set, including:

[0020] Compute feature representations of genetic data in training and validation sets.

[0021] According to the absolute position information of the SNP site on the chromosome, the feature representation is positionally embedded to obtain a feature representation with the SNP site position information.

[0022] The Transformer model was used as the deep learning architecture to construct 22 basic classifiers, and the 22 basic classifiers processed 22 autosomal data respectively.

[0023] The basic classifier is trained using feature representations with SNP site location information until the preset conditions are met to obtain a trained basic classifier.

[0024] As a preferred technical solution, the characteristic representation of the genetic data in the training set and the validation set is calculated according to the following formula:

[0025] h=W1·neHot(X)+1

[0026] X = concat(xtype ,x site )

[0027] Among them, W1 and b1 are the weight matrix and bias vector of the linear layer respectively, OneHot(·) represents the hot encoding operation, and h is the feature vector after linear transformation. type Represents the mutation type vector of the SNP site, x site represents the number vector of mutant SNP sites, concat(·) represents the merging process, and X is the feature vector after the merging process.

[0028] As a preferred technical solution, the feature representation is positionally embedded according to the absolute position information of the SNP site on the chromosome, and its expression is as follows:

[0029]

[0030] Among them, i represents the position index, j represents the dimension index of the embedding vector, pos_idx[i] represents the index of the position of the i-th SNP site on the chromosome, and embedding_dim is the dimension of the embedding vector.

[0031] As a preferred technical solution, the basic classifier is analyzed to obtain significant SNP sites, including:

[0032] Calculate the self-attention matrix of the d-th layer and the h-th head of the base classifier, and its expression is as follows:

[0033] A d,h ∈R (L+1)×(L+1)

[0034] Where L is the number of SNP loci in the input genetic data and 1 represents the CLS marker.

[0035] All self-attention matrices are averaged in the head dimension and layer dimension in turn to obtain the final self-attention matrix A of the base classifier, which is expressed as follows:

[0036]

[0037] Among them, H is the number of heads, which indicates the number of paths that the base classifier processes in parallel when processing data, and D is the number of layers, which indicates the depth of the base classifier.

[0038] Extract the association score s between the CLS marker and each SNP locus from the first row of the final self-attention matrix A l , and calculate the association score s based on the normal distribution l The p-value l , whose expression is as follows:

[0039] sl =a l ,l=1,2,…,L

[0040] a T ∈R 1×(L+1)

[0041] p l =P(X>s l ), X~N(0,1)

[0042] Among them, a T ∈R 1×(L+1) represents the row vector associated with the CLS token extracted from the self-attention matrix, each element a l Represents the association score between the CLS marker and the lth SNP site.

[0043] Given a significance level α, if p l <α, the lth SNP site is considered to be a significant SNP site.

[0044] As a preferred technical solution, the basic classifiers are integrated and learned using the test set to construct and select the optimal meta-classifier, including:

[0045] Calculate the Z-score of each base classifier on the test set.

[0046] Set several decreasing probability values ​​and use the cumulative distribution function to calculate the Z-score threshold corresponding to each probability value.

[0047] Each Z-score threshold is used in turn to screen the basic classifiers, and several basic classifier sets are generated, including a basic classifier set containing all basic classifiers.

[0048] A corresponding meta-classifier is constructed using each set of base classifiers, and the meta-classifier containing all base classifiers is used as the benchmark meta-classifier.

[0049] The AUC score of each meta-classifier is calculated, and the meta-classifier with the least number of base classifiers and the closest AUC score to the benchmark meta-classifier is selected as the optimal meta-classifier.

[0050] As a preferred technical solution, each basic classifier set is used to construct a corresponding meta-classifier, including:

[0051] The meta-classifier extracts the feature vector output by each basic classifier in the basic classifier set and performs feature fusion to obtain an integrated feature vector. The integrated feature vector is then activated in turn through a fully connected layer and a softmax activation function to obtain a classification result.

[0052] In a second aspect, the present invention further proposes a SNP site mining system based on Transformer model ensemble learning, which is applied to the SNP site mining method based on Transformer model ensemble learning as described in any scheme of the first aspect, comprising:

[0053] An acquisition module is used to acquire genetic data including chromosome SNP sites and to screen and process the genetic data;

[0054] A partitioning module, used to partition genetic data into a training set, a validation set, and a test set;

[0055] The construction module is used to build a basic classifier based on the Transformer model using the training set and the validation set;

[0056] The analysis module is used to analyze the basic classifier and obtain significant SNP sites;

[0057] The integrated learning module is used to perform integrated learning on the basic classifiers using the test set, and to construct and select the optimal meta-classifier;

[0058] The classification module is used to classify the significant SNP sites using the optimal meta-classifier and screen out the significant SNP sites related to genetic diseases.

[0059] The beneficial effects of the present invention include at least:

[0060] The present invention fully considers the biology of the data by screening and processing and reasonably dividing the genetic data, and can fully consider the biological properties and characteristics of the data and make the most of the key data information, laying the foundation for subsequent mining tasks. The basic classifier constructed based on the Transformer model uses the location information of the SNP site to reduce the demand for training sample size. The significant SNP sites are obtained by analyzing the basic classifier, and the test set is used for integrated learning to construct the optimal meta-classifier, which fully integrates the characteristic information of each classifier. Finally, the significant SNP sites are classified and processed using the optimal meta-classifier to screen out the significant SNP sites related to genetic diseases. This series of steps not only reduces the computational complexity and storage requirements of the algorithm, but also improves the mining efficiency and accuracy. At the same time, more related genetic sites can be discovered, providing strong support for genetic disease research. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Schematic diagram of the process of SNP site mining based on Transformer model ensemble learning provided in Example 1.

[0062] Figure 2This is a diagram showing the implementation framework of the SNP site mining method based on Transformer model ensemble learning provided in Example 2.

[0063] Figure 3 This is an architecture diagram of the SNP site mining system based on Transformer model ensemble learning provided in Example 3. DETAILED DESCRIPTION

[0064] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred technical solutions. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred technical solutions are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0065] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0066] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0067] Example 1

[0068] This embodiment proposes a SNP site mining method based on Transformer model ensemble learning, such as Figure 1 As shown, Figure 1 A schematic diagram of a flow chart of a SNP site mining method based on Transformer model ensemble learning provided in this embodiment, the method comprises the following steps:

[0069] S1: Obtain genetic data containing chromosome SNP sites and screen and process the genetic data.

[0070] S2: Divide the screened genetic data into training set, validation set and test set.

[0071] S3: Use the training set and validation set to build a basic classifier based on the Transformer model.

[0072] S4: Analyze the basic classifier to obtain significant SNP sites.

[0073] S5: Use the test set to perform ensemble learning on the basic classifiers to build and select the optimal meta-classifier.

[0074] S6: Use the optimal meta-classifier to classify the significant SNP sites and screen out the significant SNP sites related to genetic diseases.

[0075] It is understandable that this embodiment fully considers the biological properties and characteristics of the data and makes the most of the key data information by screening and processing the genetic data and reasonably dividing it, laying the foundation for subsequent mining tasks. The basic classifier constructed based on the Transformer model uses the location information of the SNP site to reduce the demand for training sample size. The significant SNP sites are obtained by analyzing the basic classifier, and the test set is used for integrated learning to construct the optimal meta-classifier, which fully integrates the characteristic information of each classifier. Finally, the significant SNP sites are classified and processed using the optimal meta-classifier to screen out the significant SNP sites related to genetic diseases. This series of steps not only reduces the computational complexity and storage requirements of the algorithm, but also improves the mining efficiency and accuracy. At the same time, more related genetic sites can be discovered, providing strong support for genetic disease research.

[0076] Example 2

[0077] like Figure 2 As shown, this embodiment makes improvements on the SNP site mining method based on Transformer model ensemble learning proposed in the first embodiment, such as Figure 2 As shown, Figure 2 This is a diagram of the implementation framework of the SNP site mining method based on Transformer model ensemble learning provided in this embodiment.

[0078] This embodiment downloads ADNI1 and ADNI2GO data sets from the ADNI database, and downloads the Project107 data set (hereinafter referred to as the PPMI data set) from PPMI. Genotyping data samples are obtained from the ADNI1, ADNI2GO data sets and the PPMI data sets. ADNI1, 2, GO data sets include three groups of Alzheimer's patients (AD), mild cognitive impairment (MCI) and normal controls (HC). The PPMI data set includes three groups of Parkinson's patients (PD), scans without evidence of dopaminergic deficiency (SWEDD) and normal controls (HC). SWEDD is a term used in Parkinson's disease research, referring to patients who show clinical symptoms of Parkinson's disease but do not find dopaminergic system deficiency in brain imaging scans. The initial data of the two data sets of ADNI1 and ADNI2GO are shown in Table 1.

[0079] Table 1 ADNI dataset information

[0080]

[0081]

[0082] Subsequently, the two datasets of ADNI1 and ADNI2GO were merged. The liftOver software was used to transfer the coordinates of ADNI1 from Hg18 to Hg19, so that ADNI1 became 472304 SNP sites. ADNI1 and ADNI2GO were corrected and reversed and quality controlled using the hg19 Human Genome Project reference gene as a template, and the intersection of their SNP sites was taken. Since there were repeated experimenters in the ADNI project phases 1, 2, and GO, the same samples were eliminated, and finally a mixed ADNI dataset (hereinafter referred to as the ADNI dataset) was obtained. Therefore, two datasets were obtained: the ADNI dataset and the PPMI dataset. The data of the two datasets are shown in Table 2.

[0083] Table 2 ADNI dataset and PPMI dataset information

[0084] Dataset total male female AD(PD) MCI(SWEDD) HC ADNI 1038 603 435 301 368 369 PPMI 523 343 180 325 46 152

[0085] In this embodiment, the genetic data is screened and processed, including:

[0086] The genetic data were preprocessed in sequence, including unifying coordinates, deleting outliers, and interpolating and repairing.

[0087] Calculate the p-value of the SNP site on each chromosome in the preprocessed genetic data.

[0088] For each chromosome, the first 128 SNP sites with the smallest p-value in each chromosome were extracted as the filtered genetic data.

[0089] In the specific implementation process, the genetic data were strictly quality controlled and interpolated. Plink software was used for quality control operations, and the quality control parameter settings included: deleting SNPs and individuals with a missing rate of more than 20%, setting the minimum allele frequency to 0.05, and the threshold for hard violation of Hardy-Weinberg equilibrium to 1e-6, performing independence screening to output independent SNPs, and deleting individuals with the lowest call rate in related pairs with pihat>0.2. Then the population was stratified to exclude population differences. Finally, interpolation repair was performed, and online interpolation was completed using the Michigan imputation server website (https: / / imputationserver.sph.umich.edu / ). When performing interpolation on the Michigan online interpolation website, the Minimac4 interpolation tool was used. The parameter settings are as follows: data type is unphased, reference genome version is hg19, reference panel is apps@1000g-phase-3-v5(hg19), population group is European (eur), phasing tool is eagle, mode is imputation, and the Rsq filter value is set to 0.3.

[0090] The method for processing the ADNI dataset is as follows: Since the ADNI data comes from Europe, non-European samples are removed. First, the human reference genome and the ADNI dataset of the hg19 coordinates are quality controlled. Then merge them, because the previous two sample sets ADNI1 and ADNI2GO were originally merged according to this standard, so the reference genome will not lose any sites when merged with the ADNI dataset. Then merge to do PCA population distribution, generate a multidimensional scaling analysis diagram, and display the distribution of the first 10 dimensions, and finally select 940 samples close to the European samples. After interpolation and re-quality control, 5361007 variable sites and 940 samples were obtained.

[0091] The method for processing the PPMI dataset is as follows: Since the PPMI data comes from Europe, non-European samples are removed. First, the dataset is converted from hg18 coordinates to hg19 coordinates using liftOver software. In the second step, the human reference genome and PPMI dataset of hg19 coordinates are quality controlled. Then they are merged, and then the PCA population distribution is merged to generate a multidimensional scaling analysis diagram, and the distribution of the first 10 dimensions is displayed. Finally, 461 samples close to European samples are selected. After interpolation and re-quality control, 1,192,960 variable sites and 461 samples are obtained.

[0092] Then the two datasets are processed, MCI samples and AD samples are merged into one class, called abnormal samples (Disease), SWEDD and PD samples are also merged into one class (Disease), and then the downstream tasks are carried out. The samples of the obtained dataset after preprocessing are shown in Table 3.

[0093] Table 3 Samples after preprocessing of ADNI dataset and PPMI dataset

[0094] Dataset Total number of samples Number of variable sites Disease HC ADNI 940 5361007 612 328 PPMI 461 1192960 329 132

[0095] When initially screening the genetic data, plink software was used to perform association analysis on the preprocessed data, and a logistic model was used for logistic regression analysis to calculate the p-value of each variable site, and the first 128 variable sites with the smallest p-value on each chromosome were extracted as data for subsequent tasks.

[0096] In this embodiment, a stratified random sampling strategy is used to divide the data set, and the entire data set is divided into a training set, a validation set, and a test set at a ratio of 3:1:1. During the division process, it is ensured that the ratio of disease samples and normal samples between each data set is consistent. In addition, the training set and the validation set are subjected to four-fold cross validation, and four groups of data are finally generated. The test set is an independent data set, does not participate in any training, and is only used for the final model performance test. In this embodiment, 187 samples are divided from the ADNI data set (a total of 940 samples) as an independent test set, which does not participate in any training and is only used for the final model performance test. The remaining 753 samples are subjected to 4-fold cross validation to obtain the original training set and the original validation set of each fold. Therefore, each fold original training set contains 563 samples, the original validation set contains 187 samples, and the test set contains 187 samples. The sample distribution of the original training set and the original validation set of each fold is shown in Table 4, and the sample distribution of the original test set is shown in Table 5.

[0097] Table 4 Distribution of samples in the original training set and the original validation set

[0098]

[0099] Table 5 Distribution of original test set samples

[0100]

[0101] In order to improve the model's ability to characterize and understand data and find more variable sites related to diseases, a basic classifier is constructed based on chromosomes by dividing chromosomes. Then, the variable sites related to diseases are found by training the basic classifier and interpreting the data learning results with the self-attention matrix. This embodiment uses sine-cosine position encoding technology and a representation method specific to the data type to characterize the data, and uses Transformer as the basic architecture of the basic model to construct a solution for finding relevant variable sites.

[0102] In this embodiment, a basic classifier based on the Transformer model is constructed using a training set and a validation set, including:

[0103] Compute feature representations of genetic data in the training and validation sets:

[0104] h=W1·OneHot(X)+b1

[0105] X = concat(x type ,x site )

[0106] Among them, W1 and b1 are the weight matrix and bias vector of the linear layer respectively, OneHot(·) represents the hot encoding operation, and h is the feature vector after linear transformation. type Represents the mutation type vector of the SNP site, x site represents the vector of the number of mutant SNP sites, concat(·) represents the merging process, and X is the feature vector after the merging process, which means that they are merged into one event, and there are 40 cases in total.

[0107] Given that the classification model architecture consists of 22 basic classifiers, each classifier corresponds to a chromosome, so the position information of the variable site on the chromosome can be used as a position embedding feature. This embodiment uses the absolute position information of the SNP site on the chromosome and performs position embedding based on the sine and cosine functions. In order to facilitate the encoding of position information in the deep learning neural network architecture, this embodiment adopts a method of embedding positions in a sequence. According to the absolute position information of the SNP site on the chromosome, the feature representation is positionally embedded to obtain a feature representation with the SNP site position information, and the position information is provided for each position in the input sequence. The expression is as follows:

[0108]

[0109]

[0110] Among them, i represents the position index, j represents the dimension index of the embedding vector, pos_idx[i] represents the index of the position of the i-th SNP site on the chromosome, and embedding_dim is the dimension of the embedding vector.

[0111] In this embodiment, Transformer is used as a deep learning architecture to construct a chromosome-based basic classifier, and a total of 22 basic classifiers are established, and the 22 basic classifiers correspond to processing 22 autosomal data respectively. According to the training curve, appropriate hyperparameter training is selected to ensure that all basic classifiers reach the degree of convergence. The model architecture of the basic classifier of this embodiment is specifically described as follows:

[0112] Each base classifier corresponds to a chromosome, ensuring that each base classifier processes data from a single chromosome (therefore, there are a total of 22 base classifiers corresponding to 22 autosomes). The input includes the genotype of each SNP site (n=128, the first 128 SNP sites with the smallest chromosome p-value) and its position on the corresponding chromosome. These two types of information are processed separately. For the genotype data of each site, the input dimension is 40, which is expanded to 512 dimensions through word embedding. The position information of each SNP site on the corresponding chromosome continues to be processed using the position embedding method to map it to a 512-dimensional space. Then, the processed data from the two information sources are combined by addition and connected with the classification label for classification. Next, the data is processed by a self-attention mechanism consisting of 12 heads and an 8-layer structure. Finally, a 512x2 fully connected layer is used to generate binary classification probabilities (healthy individuals or patients).

[0113] The basic classifier is trained using the feature representation with SNP site location information until the preset conditions are met to obtain a trained basic classifier. The loss function uses the cross entropy loss function, and the formula is as follows:

[0114]

[0115] The training set and validation set are used to train and select the model. Each column represents a chromosome, and each row represents the AUC score of the corresponding validation set in the four-fold cross validation. The basic classifier structure is the same, and the initial learning rate is 5e-5. In 300 training cycles, the CosineAnnealingLR learning rate decay strategy is used to adjust the learning rate. The optimizer uses Adam. The final results are shown in Tables 6 and 7.

[0116] Table 6 Results of the basic classifier on the ADNI validation set

[0117]

[0118] Table 7 Results of the basic classifier on the PPMI validation set

[0119]

[0120] In this embodiment, the basic classifier is analyzed to obtain significant SNP sites, including:

[0121] Calculate the self-attention matrix of the d-th layer and the h-th head of the base classifier, and its expression is as follows:

[0122] A d,h ∈R (L+1)×(L+1)

[0123] Where L is the number of SNP loci in the input genetic data and 1 represents the CLS marker.

[0124] All self-attention matrices are averaged in the head dimension and layer dimension in turn to obtain the final self-attention matrix A of the base classifier, which is expressed as follows:

[0125]

[0126] Among them, H is the number of heads, which indicates the number of paths that the base classifier processes in parallel when processing data, and D is the number of layers, which indicates the depth of the base classifier.

[0127] Extract the association score s between the CLS marker and each SNP locus from the first row of the final self-attention matrix A l , and calculate the association score s based on the normal distribution l The p-value l , whose expression is as follows:

[0128] s l = l ,=1,2,…,

[0129] a T ∈R 1×(L+1)

[0130] p k =(X>s l ), X~N(0,1)

[0131] Among them, a T ∈R 1×(L+1) represents the row vector associated with the CLS token extracted from the self-attention matrix, each element a l Represents the association score between the CLS marker and the lth SNP site.

[0132] Given a significance level α, if p l<α, the lth SNP locus is considered to be a significant SNP locus. α is 0.0001 or 0.05.

[0133] In the specific implementation process, α is set to 0.0001 for identifying highly significant SNP sites, and α is set to 0.05 for identifying relatively loose significant SNP sites.

[0134] For p l Significant SNP sites with a p-value < 0.0001 are considered highly significant and are directly reported as the main research focus.

[0135] For 0.0001≤p l <0.05 significant SNP sites, although these sites are not as good as p l Significant SNP sites with a value of <0.0001 are less significant but still statistically significant and are mainly used in subsequent pathway analysis to reveal broader biological processes.

[0136] In this embodiment, p l The significant SNP sites with a value <0.0001 are listed, and the corresponding associated genes retrieved from the dbSNP database. Genes that have been reported to be associated with diseases are emphasized in bold font. The results are shown in Tables 8 and 9.

[0137] Table 8 Results of the significant SNP sites identified on each chromosome (ADNI dataset)

[0138]

[0139]

[0140] Table 9 Results of the significant SNP sites identified on each chromosome (PPMI dataset)

[0141]

[0142]

[0143] In this embodiment, the basic classifiers are integrated and learned using the test set to construct and select the optimal meta-classifier, including:

[0144] Calculate the Z-score of each base classifier on the test set.

[0145] Set several decreasing probability values ​​and use the cumulative distribution function to calculate the Z-score threshold corresponding to each probability value.

[0146] Each Z-score threshold is used in turn to screen the basic classifiers, and several basic classifier sets are generated, including a basic classifier set containing all basic classifiers.

[0147] The AUC score of each meta-classifier is calculated, and the meta-classifier with the least number of base classifiers and the closest AUC score to the benchmark meta-classifier is selected as the optimal meta-classifier.

[0148] In the specific implementation process, the Z-score method is used to distinguish the grades of the basic classifiers, and the grades are divided into six grades according to the one-sided test interval. Then the basic classifiers included in these six grades are integrated and learned to construct 6 meta-models. The meta-models are trained with the training set and the validation set to obtain six meta-models. Finally, the meta-model with less data usage and better effect on the test set is selected, and the corresponding chromosome is used as the key chromosome for subsequent biological pathway mining.

[0149] Then, the metamodel method with less data usage and better effect on the test set is relatively novel. The AUC results of each metamodel on the test set are compared, and the best model that uses the least amount of data to express all the data is selected. Specifically, the best model should meet two conditions: Condition 1, the ratio of the model's AUC score to the metamodel AUC using all chromosomes is higher than 98%. Condition 2, on the basis of meeting condition 1, select the metamodel with the least amount of data.

[0150] The meta-training strategy is as follows: each meta-classifier is trained for 50 cycles, using the Adam optimizer, an initial learning rate of 5e-5, and a CosineAnnealingLR learning rate decay strategy. The p-value is obtained by performing a one-tailed hypothesis test on the AUC score of the base classifier. After training, the results of the test set are shown in Tables 10 and 11. The upper table in Tables 10 and 11 shows the AUC score of the meta-model for each chromosome set on the test set, and the lower table shows the chromosome number contained in each chromosome set.

[0151] Table 10 Performance results of the meta-model on the test set (ADNI dataset)

[0152]

[0153] Threshold Selected Chromosomes p≥0.5 1，2，3，4，5，6，7，9，11，12，14，17，19，20 p≥0.4 1，2，3，4，5，6，7，9，11，12，14，17，18，19，20 p≥0.3 1，2，3，4，5，6，7，9，11，12，14，17，18，19，20，22 p≥0.2 1，2，3，4，5，6，7，8，9，11，12，14，15，17，18，19，20，22 p≥0.1 1，2，3，4，5，6，7，8，9，10，11，12，14，15，16，17，18，19，20，22 all chr 1，2，3，4，5，6，7，8，9，10，11，12，13，14，15，16，17，18，19，20，21，22

[0154] Table 11 Performance results of the meta-model on the test set (PPMI dataset)

[0155]

[0156] Threshold Selected Chromosomes p≥0.5 1，5，7，10，11，12，14，16，18，19，20 p≥0.4 1，3，5，7，10，11，12，14，16，18，19，20 p≥0.3 1，2，3，5，7，9，10，11，12，14，16，18，19，20 p≥0.2 1，2，3，5，6，7，8，9，10，11，12，14，16，18，19，20，22 p≥0.1 1，2，3，5，6，7，8，9，10，11，12，13，14，16，17，18，19，20，21，22 all chr 1，2，3，4，5，6，7，8，9，10，11，12，13，14，15，16，17，18，19，20，21，22

[0157] Through the performance test of the meta-model, the key chromosomes of the two data sets are finally locked. For the ADNI data set, P>=0, the chromosome set of 5 meets the selection condition. For the PPMI data set, P>=0, the chromosome set of 1 meets the selection condition.

[0158] This example tests the performance of ensemble learning and non-ensemble learning methods on the classification task. In order to verify the effectiveness of ensemble learning on model performance, ensemble learning is set aside and the concepts of meta-model and base classifier are excluded. Instead, the same architecture as the base classifier is used to train and evaluate a total of 2816 extraction positions (2816=22x128), which is a non-ensemble learning method. Here, the position information embedding method selected is trainable position embedding. The training set, validation set, and test set are consistent with the previous settings.

[0159] Due to the large number of computation locations, the batch size of the samples is set to 2 to ensure that the model can converge considering the available GPU memory limitations. The final results are shown in Table 12. Obviously, the model using ensemble learning outperforms the model without ensemble learning.

[0160] Table 12 Results of testing the ADNI dataset using ensemble learning and non-ensemble learning methods

[0161]

[0162] In the process of using each set of basic classifiers to construct the corresponding meta-classifier, the weight of each basic classifier is fixed, and the feature output of the penultimate layer of the basic classifier is extracted, and the feature output is regarded as the input of the meta-model. Each basic classifier is responsible for an autosome, and CLStoken is a summary of the chromosome data for each basic classifier. Therefore, CLStoken has information that is helpful in distinguishing normal samples from diseased samples. Integrating the summaries obtained from each basic classifier is equivalent to integrating the information of the 22 autosomes that is useful for classification. Take this integrated information as the input of the meta-model and use meta-model ensemble learning.

[0163] The integrated learning meta-model component of this embodiment adopts a simple average fusion strategy, and the specific steps are as follows:

[0164] (1) Weight allocation: Assume there are M basic classifiers. Define M weight parameters W m , where m = 1, 2, ..., M. The weight parameter of each base classifier is the same, and the value is 1 / M, that is, for any base classifier m, its weight parameter W m =1 / M.

[0165] (2) Feature fusion: The penultimate feature map of the mth base classifier is represented as a vector hm . m And the corresponding weight W m Multiply: Z m =W m *h m , for any m, connect the M weighted feature vectors to get the integrated feature: Z = [Z1, Z2, ..., Z M ].

[0166] (3) Final classification: Z is input into a fully connected layer through a softmax activation function to obtain the binary classification probability, which is expressed as:

[0167] P = Softmax(W z *X+β)

[0168] Where Wz and β are learnable weight matrices and biases. P is the output vector, where [P1, P2] represents the classification probability.

[0169] In this embodiment, after obtaining the significant SNP site, the method further includes:

[0170] Genes associated with significant SNP sites were pooled to obtain a gene set.

[0171] Perform enrichment analysis on gene sets and construct biological pathways related to genetic diseases.

[0172] In the specific implementation process, the R language software package clusterProfiler was used to calculate the enrichment p value of each pathway term in the gene set G contained in the key chromosome based on the hypergeometric distribution and select the significant pathway (p<0.05). The specific formula for calculating the hypergeometric distribution is as follows:

[0173]

[0174] Among them, N is the number of background genes, M is the number of pathway genes, n is the number of test genes, and m is the number of test genes included in the pathway. The significant pathway results are shown in Table 12. Pathways that have been reported to be related to diseases are emphasized in bold fonts.

[0175] Table 12 Significant biological pathways obtained from the two datasets

[0176]

[0177] Example 3

[0178] like Figure 3As shown, this embodiment proposes a SNP site mining system based on Transformer model ensemble learning, which is applied to the SNP site mining method based on Transformer model ensemble learning as described in the above embodiment, including: an acquisition module 100, a division module 200, a construction module 300, a parsing module 400, an ensemble learning module 500 and a classification module 600.

[0179] Among them, the acquisition module 100 is used to obtain genetic data containing chromosome SNP sites and screen the genetic data. The division module 200 is used to divide the genetic data into a training set, a validation set and a test set. The construction module 300 is used to use the training set and the validation set to construct a basic classifier based on the Transformer model. The parsing module 400 is used to parse the basic classifier to obtain significant SNP sites. The integrated learning module 500 is used to perform integrated learning on the basic classifier using the test set, and construct and screen out the optimal meta-classifier. The classification module 600 is used to classify and process the significant SNP sites using the optimal meta-classifier, and screen out significant SNP sites related to genetic diseases.

[0180] It should be noted that the aforementioned explanation of the embodiment of the SNP site mining method based on Transformer model ensemble learning is also applicable to the SNP site mining system based on Transformer model ensemble learning in this embodiment, and will not be repeated here.

[0181] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0182] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0183] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0184] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0185] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0186] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A SNP site mining method based on Transformer model ensemble learning, characterized in that: include: Obtain genetic data containing chromosome SNP sites, and screen and process the genetic data; Divide the screened genetic data into training set, validation set and test set; Use the training set and validation set to build a basic classifier based on the Transformer model; Analyze the basic classifier to obtain significant SNP sites; Use the test set to perform ensemble learning on the basic classifiers, build and select the optimal meta-classifier; The optimal meta-classifier is used to classify the significant SNP sites and screen out the significant SNP sites related to genetic diseases.

2. The SNP site mining method based on Transformer model ensemble learning according to claim 1, characterized in that: Screening and processing of genetic data, including: The genetic data were preprocessed in sequence, including unifying coordinates, deleting outliers, and interpolation repair; Calculate the p-value of the SNP site on each chromosome in the preprocessed genetic data; For each chromosome, the first 128 SNP sites with the smallest p-value in each chromosome were extracted as the filtered genetic data.

3. The SNP site mining method based on Transformer model ensemble learning according to claim 1, characterized in that: After obtaining the significant SNP site, the method further comprises: Genes associated with significant SNP sites were pooled to obtain a gene set; Perform enrichment analysis on gene sets and construct biological pathways related to genetic diseases.

4. The SNP site mining method based on Transformer model ensemble learning according to claim 1, characterized in that: Using the training set and validation set, we build a basic classifier based on the Transformer model, including: Compute feature representations of genetic data in training and validation sets; According to the absolute position information of the SNP site on the chromosome, the feature representation is positionally embedded to obtain a feature representation with the SNP site position information; The Transformer model is used as the deep learning architecture to construct 22 basic classifiers, which process 22 autosomal data respectively. The basic classifier is trained using feature representations with SNP site location information until the preset conditions are met to obtain a trained basic classifier.

5. The SNP site mining method based on Transformer model ensemble learning according to claim 4, characterized in that: The feature representation of genetic data in the training set and validation set is calculated according to the following formula: h=W1·OneHot(X)+b1 X=concat(x type ,x site ) Among them, W1 and b1 are the weight matrix and bias vector of the linear layer respectively, OneHot(·) represents the hot encoding operation, and h is the feature vector after linear transformation; x type Represents the mutation type vector of the SNP site, x site represents the number vector of mutant SNP sites, concat(·) represents the merging process, and X is the feature vector after the merging process.

6. The SNP site mining method based on Transformer model ensemble learning according to claim 4, characterized in that: According to the absolute position information of the SNP site on the chromosome, the feature representation is positionally embedded, and its expression is as follows: Among them, i represents the position index, j represents the dimension index of the embedding vector, pos_idx[i] represents the index of the position of the i-th SNP site on the chromosome, and embedding_dim is the dimension of the embedding vector.

7. The SNP site mining method based on Transformer model ensemble learning according to claim 4, characterized in that: The basic classifier was analyzed to obtain significant SNP sites, including: Calculate the self-attention matrix of the d-th layer and the h-th head of the base classifier, and its expression is as follows: A d,h ∈R (L+1)×(L+1) Where L is the number of SNP sites in the input genetic data, and 1 represents the CLS marker; All self-attention matrices are averaged in the head dimension and layer dimension in turn to obtain the final self-attention matrix A of the base classifier, which is expressed as follows: Among them, H is the number of heads, which indicates the number of paths that the base classifier processes in parallel when processing data, and D is the number of layers, which indicates the depth of the base classifier; Extract the association score s between the CLS marker and each SNP locus from the first row of the final self-attention matrix A l , and calculate the association score s based on the normal distribution l The p-value l , whose expression is as follows: s1=a l ,l=1,2,…,L a T ∈R 1×(L+1) p l =P(X>s l ),X~N(0,1) Among them, a T ∈R 1×(L+1) represents the row vector associated with the CLS token extracted from the self-attention matrix, each element a l represents the association score between the CLS marker and the lth SNP site; Given a significance level α, if p l <α, the lth SNP site is considered to be a significant SNP site.

8. The SNP site mining method based on Transformer model ensemble learning according to claim 4, characterized in that: Use the test set to perform ensemble learning on the basic classifiers, build and select the optimal meta-classifier, including: Calculate the Z-score of each base classifier on the test set; Set several decreasing probability values ​​and use the cumulative distribution function to calculate the Z-score threshold corresponding to each probability value; Each Z-score threshold is used in turn to screen the basic classifiers, generating several basic classifier sets, including a basic classifier set containing all basic classifiers; Use each set of basic classifiers to build a corresponding meta-classifier, and use the meta-classifier containing all basic classifiers as the benchmark meta-classifier; The AUC score of each meta-classifier is calculated, and the meta-classifier with the least number of base classifiers and the closest AUC score to the benchmark meta-classifier is selected as the optimal meta-classifier.

9. The SNP site mining method based on Transformer model ensemble learning according to claim 8, characterized in that: Use each base classifier set to build a corresponding meta classifier, including: The meta-classifier obtains an integrated feature vector by extracting the feature vector output by each basic classifier in the basic classifier set and performing feature fusion; the integrated feature vector is then activated in turn through a fully connected layer and a softmax activation function to obtain a classification result.

10. A SNP site mining system based on Transformer model ensemble learning, characterized in that: include: An acquisition module is used to acquire genetic data including chromosome SNP sites and to screen and process the genetic data; A partitioning module, used to partition genetic data into a training set, a validation set, and a test set; The construction module is used to build a basic classifier based on the Transformer model using the training set and the validation set; The analysis module is used to analyze the basic classifier and obtain significant SNP sites; The integrated learning module is used to perform integrated learning on the basic classifiers using the test set, and to construct and select the optimal meta-classifier; The classification module is used to classify the significant SNP sites using the optimal meta-classifier and screen out the significant SNP sites related to genetic diseases.

Images