Method for screening of genetic loci based on DNA methylation

Abstract

The application provides a DNA methylation-based gene point site screening method, which comprises the following steps: 1) obtaining initial parameters of a gene point site screening model through meta-learning; and 2) retraining the gene point site screening model based on the initial parameters obtained in step 1). The method provided by the application can alleviate the overfitting phenomenon during training of the gene point site screening model.

Description

TECHNICAL FIELD

[0001] The present application relates to a method for screening gene point positions, and particularly relates to a method for screening gene point positions based on DNA methylation. BACKGROUND

[0002] Schizophrenia is a mental illness characterized by hallucinations, delusions, disorganized thinking and cognitive deficits, which can affect a person's thoughts, feelings and behavior. The symptoms of schizophrenia develop gradually or suddenly, vary from patient to patient, and develop in cycles of remission and relapse. Over time, the mental function and social relationship of a patient with schizophrenia will slowly decline, which will lead to significant personality changes, social isolation, occupational barriers, cognitive barriers and poor health in the patient, and is prone to lead to suicidal behavior and violent attacks on others.

[0003] At present, the cause of schizophrenia is not clear, and some medical research believes that the interaction between genes and a series of environmental factors may lead to the occurrence of schizophrenia. Therefore, many studies have been conducted on the influence of epigenetics on the formation of schizophrenia. DNA methylation is one of the most deeply studied epigenetic modifications, which is involved in many cellular regulatory processes. At present, it can be collected through high-throughput chips. More importantly, the main gene positions related to schizophrenia can be mined by selecting features. At the same time, the DNA methylation level is reversible, and medical experts can then study drugs that reverse DNA methylation to treat schizophrenia.

[0004] In addition, DNA methylation is one of the important reasons for the occurrence of many cancers. Screening important gene positions through DNA methylation data can provide an important basis for the treatment of cancer.

[0005] DNA methylation data is a typical high-dimensional small sample data. First, although the high dimension of the data covers a lot of key information, there is also a lot of redundant and non-key information in the high-dimensional data, which makes feature selection more difficult. Second, too few samples can easily reduce the performance of machine learning algorithms, and the model will have overfitting phenomenon. The patent with publication number CN112927757A uses statistical methods such as differential analysis to select features of methylation data, but the invention only focuses on extracting as much information as possible from the data, ignoring the influence of overfitting on the final effect. SUMMARY

[0006] In view of the above defects of the prior art, the present application provides a method for screening gene point positions based on DNA methylation, which solves the problem of overfitting of the gene point position screening model during training.

[0007] To solve the above problems, the technical scheme adopted by the present application is: a gene point screening method based on DNA methylation, the method comprising:

[0008] 1) Obtain the initialization parameters of the gene point screening model through meta-learning;

[0009] 2) Re-train the gene point screening model based on the initialization parameters obtained in step 1).

[0010] Preferably, the method further comprises: preprocessing the DNA methylation data.

[0011] Preferably, the preprocessing comprises: cleaning processing, and / or quantile standardization processing, and / or segmentation processing.

[0012] Preferably, the gene point screening model comprises a weight gate layer, a fully connected layer and a softmax layer.

[0013] Preferably, the fully connected layer is four layers.

[0014] Preferably, in step 1), the obtained initialization parameters are the parameters of the fully connected layer; in step 2), only the parameters of the weight gate layer are updated when re-training the point screening model.

[0015] Preferably, the method further comprises: adding the L1 norm of the weight gate layer to obtain a sparse result, and adding the L2 norm of the parameters of the fully connected layer to limit the complexity of the model.

[0016] Preferably, the gene point is a schizophrenia gene point.

[0017] Preferably, the method further comprises: classifying the obtained feature selection result.

[0018] Preferably, the classification processing is: compressively encoding the feature selection result and inputting it into a linear SVM classifier for classification.

[0019] The present application has the beneficial effects that: the present application uses DNA methylation data, trains the model using the meta-learning method to obtain the initialization parameters, and uses the initialization parameters as the initialization parameters of the feature selection network based on the attention mechanism, and then performs gene point screening, which relieves the overfitting phenomenon, and provides a more effective feature selection method.

[0020] The concept, specific structure and technical effects of the present application will be further described below with reference to the accompanying drawings, so as to fully understand the purpose, features and effects of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1A flow chart of one preferred embodiment of the DNA methylation-based gene locus screening method provided by the present application is shown.

[0022] Figure 2 For Figure 1 A structural schematic diagram of the gene locus screening model used in the method shown. DETAILED DESCRIPTION

[0023] Figure 1 One preferred embodiment of the DNA methylation-based gene locus screening method provided by the present application is shown.

[0024] As Figure 1 shown, the method of this embodiment includes: 1) DNA methylation data acquisition; 2) data preprocessing; 3) meta-learning to obtain the initialization parameters of the locus screening model; and 4) key gene locus screening. After screening, subsequent processing can be performed.

[0025] The following will describe this embodiment in detail.

[0026] The data used in this embodiment comes from the TCGA database.

[0027] TCGA is a cooperative project launched by the U.S. Cancer Institute and the National Human Genome Research Institute in 2005. The project aims to find disease-related death genes using genomic analysis techniques, understand the mechanism of cancer occurrence, and help improve diagnostic methods and treatment standards. The TCGA dataset contains 9652 DNA methylation data samples from 33 types of human cancers. The DNA methylation data collected in this dataset has more than 450,000 features, and the average public dataset also has about 400,000 features.

[0028] TCGA methylation data needs to be preprocessed before further use.

[0029] First, there are many missing features in TCGA, and the data needs to be cleaned. Specifically, missing data can be roughly divided into two categories: high missing values and low missing values. The processing of these two types needs to be distinguished. For high missing values, the feature is discarded; for low missing values, since there are fewer missing values under the same feature, the average value is used to replace the missing data.

[0030] Second, the original data is subject to a typical non-Gaussian distribution. If the feature selection and dimension reduction algorithm under the Gaussian assumption is used, it will lead to poor results. At the same time, the present application only focuses on whether the feature point is in a high methylation state or a low methylation state, which means that the quantile normalization method widely used in biological science can be applied, preserving the original non-Gaussian distribution of the data. This standardization method uses the mean value of the coordinate data point to replace the original value. These data can maintain the same distribution as the original as much as possible.

[0031] Third, the DNA methylation data is too large in dimension, and it is complex to directly select features from the original data. This is because the dimension of each sample is very high, and the excellent classification ability of the neural network is based on its deep network structure, which also means that there are more parameters in the neural network. High-dimensional input and deeper network are prone to cause memory overflow. Therefore, when performing feature selection, the DNA methylation data needs to be segmented, and the network is trained in batches, and a unified measurement standard is used to select features. At the same time, considering that the MAML algorithm used is a gradient-based algorithm, it also has very high time and space complexity. Therefore, the features are segmented and processed, and the methylation data is divided into blocks composed of several gene points that need to be retained. This operation can greatly reduce the time required for training.

[0032] After preprocessing the TCGA methylation data, the initialization parameters of the point position screening model can be obtained through the meta-learning algorithm.

[0033] The maximum number of samples in a single class in the TCGA methylation data set is less than 1000, and the minimum number of samples is even only 10. There are obstacles in using transfer learning on this problem. Transfer learning needs enough data to use the information of related tasks. However, the data volume of TCGA cannot meet the requirements.

[0034] Meta-learning is another way to transfer information. From the perspective of meta-learning, the original data set is reconstructed to obtain the source task and the target task composed of the support set and the validation set. According to this, enough source tasks can be constructed from the 33 categories in TCGA. These multi-task patterns make up for the problem of insufficient data, which is the reason why the present application chooses meta-learning as the basic algorithm.

[0035] For the site screening model, we adopted the attention-based model proposed by Minmin Zhang (Minmin Zhang, Changchun Pan, Haichun Liu, Qinting Zhang, and Haozhe Li. An attention-based deep learning method for schizophrenia patients classification using dna methylation data. In 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pages 172-175. IEEE, 2020.). They proposed a neural network based on attention mechanism for feature selection. As shown in Figure 2 Figure 1, this network has three parts: the weight-gated layer, the fully connected layer, and the softmax layer. The weight-gated layer means that each neuron in this layer is wrapped with a sigmoid function. The goal is to measure the importance of each feature using this attention mechanism and perform feature selection. The fully connected layer is used to increase the complexity of the model so that it can properly handle the data. However, the number of layers cannot be chosen arbitrarily to avoid overfitting or underfitting. Four layers of fully connected layers are the best choice here, so a four-layer fully connected network is built here as an intermediary. The last softmax layer is used to create a loss function and perform gradient backpropagation.

[0036] To get more convincing results, the loss function of the model can also be modified based on various considerations here. For example, the L1 norm of the weight-gated layer can be added to get sparse results, and the L2 norm of the rest of the parameters can be added to limit the complexity of the model. The results of these two restrictions are discussed below.

[0037] Most of the loss functions in deep learning can be written in the form of equation (1) below:

[0038]

[0039] The term Ω(θ) is usually set as the restriction condition of the model, and the purpose of choosing the L2 norm here is to control the parameters within a more reasonable range to avoid overfitting. To get sparse results and perform feature selection on the weight-gated layer, another L1 norm is added to the weight-gated layer here. In general, the maximum likelihood of binary classification estimates L(w) as shown in equation (2) below:

[0040]

[0041] Here, if f(w j ) is a Gaussian prior with zero mean, the maximum log-likelihood of L(w) is represented by the following equation (3):

[0042]

[0043] This is exactly the statistical interpretation of the L2 norm. Obviously, the L2 norm limits the parameters of the model to avoid too large outliers. Similarly, when the prior distribution is Laplace distribution, the following equation (4) can be obtained:

[0044]

[0045] The L1 norm regularization not only limits the parameter space, but also brings sparsity. The probability of the value close to 0 of the Laplace distribution is much larger than that of the normal distribution, and the probability of getting extreme values is also larger than that of the normal distribution. This means that, compared with the normal distribution, the parameters with Laplace prior tend to get extreme values close to 0 or more. While the normal distribution prior brings smooth values. This phenomenon can also be explained from the perspective of optimization. The contour of L1 norm is a sharp diamond, while the contour of L2 norm is a smooth hypersphere space, so the parameters of the loss function are easily at the endpoints of the L1 norm contour in the optimization process, which can also provide sparse coding of the original features. Therefore, the softmax loss is modified as follows:

[0046]

[0047] In the loss function, N represents the total number of samples, and M represents the number of sample categories. σ(w) represents the sigmoid function, w is the parameter of the weight gating layer, and θ is the rest of the parameters. The output of the weight gating layer is the Hadamard product of the sigmoid function σ(w) and the features. In order to obtain the sparse result of the weight gating layer, an L1 norm of the parameter is added in the weight gating layer here. The L1 norm is the sparsest among all convex norms, which means that most of the values of σ(w) of the trivial features tend to 0, and only a few features have a greater impact on the final classification result, often close to 1. At the same time, an L2 norm is added to the parameters of the fully connected layer to alleviate overfitting.

[0048] During backpropagation, good prediction results are gradually obtained here. The sparse result of the weight gating layer can also be obtained here. However, due to the randomness caused by random initialization of parameters, in order to obtain more robust feature selection, multiple repeated training and cross-validation can be performed. Feature selection is mainly based on the final value and accuracy of the weight gating layer, using the method of scoring features, and then selecting important features according to the score.

[0049] MAML algorithm is an optimization-based meta-learning method that can utilize information in related tasks. The basic training unit of meta-learning is a training task, and the dataset of each training task is divided into a support set and a query set. MAML is a double-optimization algorithm that utilizes as much prior information as possible from multi-task data acquisition. In this process, the support set is usually used to mine task-specific information, and the query set always acquires shared knowledge between tasks.

[0050] The processed TCGA sample has about 400,000 features. In order to avoid the trouble of dimensionality reduction, the data is divided into 5000 feature groups. The algorithm will be repeated 100 times or more and cross-validated to obtain good results.

[0051] At the same time, considering that there may be significant differences in feature selection results between different tasks in TCGA, if feature selection is performed at the same time, it is easy to fall into a poor solution. Therefore, when using MAML, the structure of the model needs to be adjusted. All weight gated layers can be set to constants and remain unchanged to reduce the disruptive effect of differences between TCGA-constructed tasks. During the training process of the MAML algorithm, only the fully connected layer is trained to obtain the initial parameters. Then load these parameters into the complete model and train them using subsequent data to obtain feature selection and classification results.

[0052] Specifically, a number of corresponding training tasks are constructed using the preprocessed TCGA dataset, and the neural network is trained according to the forward and backward propagation algorithm in the neural network. Meta-learning is performed using multiple training tasks divided from the processed TCGA dataset, and during the training process, the parameters of all weight gated layers are fixed as constants and do not participate in gradient backpropagation parameter update. This is mainly to give all features the same attention and focus on training an effective classifier. A loss function is constructed, and the hyperparameters of the model network are adjusted using the stochastic gradient descent algorithm to minimize the loss function as much as possible. After multiple iterations, a set of effective model initialization parameters is obtained. The algorithm for constructing tasks in meta-learning is as follows:

[0053]

[0054] The selected loss function for model training in meta-learning is:

[0055]

[0056] Further, the algorithm for obtaining model initialization parameters in meta-learning is as follows:

[0057]

[0058]

[0059] The above algorithm shows how to obtain pre-trained parameters using MAML and alleviate the overfitting problem using the information of the TCGA dataset.

[0060] By training the TCGA methylation data using the above MAML algorithm, the initialization parameters can be obtained. Through experiments, a new model is retrained using the obtained initialization parameters, and the results show that only less than 11 steps of gradient descent are needed to obtain reasonable results. Therefore, the parameters provided by MAML can be easily generalized to similar tasks.

[0061] In addition, in order to reasonably analyze the experimental results, the present application also uses cross-validation and multiple repeated experiments. The divided TCGA data is used to train the task, the full connection layer is trained according to MAML, and further classification is performed. Then, the results of randomly initializing the parameters of the full connection network and directly using the original features are compared. The results are shown in Table 1:

[0062] Table 1 Accuracy of the classifier

[0063]

[0064] As can be seen from Table 1, the model using MAML achieves the highest accuracy, because it can gradually discover the common information between tasks. This is also the reason why the result is better than random initialization.

[0065] After obtaining the initialization parameters of the point screening model, the parameters can be loaded into the complete model, and the parameters of the fully connected layer are frozen. Here, only the weight gate layer is trained and updated, and only the preprocessed limited DNA methylation data is used to update the weight gate layer parameters of the neural network through gradient backpropagation, for feature selection. According to the values of the weight gate layer and the classification results, important gene sites are screened out. The obtained feature selection results can be used for subsequent processing, for example, the feature selection results can be compressed and encoded and input into a linear SVM classifier for classification.

[0066] When retraining, DNA data of a specific disease can be used for training to screen out important gene sites, for example, data of a certain cancer. Through experiments, the present inventors have found that DNA data of schizophrenia can also be used to screen out important gene sites.

[0067] The preferred embodiments of the present application have been described above in detail. It should be understood that modifications and variations to the present application can be affected by those skilled in the art without departing from the scope of the application. Accordingly, it is intended that all possible modifications and alterations be included within the scope of the present application as defined by the following claims.

Claims

1. A DNA methylation-based genetic point site screening method, characterized by, The method comprises: 1) obtaining initial parameters of a gene point screening model by a MAML meta-learning algorithm; 2) retraining the gene point screening model based on the initial parameters obtained in step 1); The gene point screening model is a model based on attention; The DNA methylation data is preprocessed; the preprocessing comprises: cleaning processing, quantile standardization processing and segmentation processing; The gene point screening model comprises a weight gate layer, a fully connected layer and a softmax layer; the fully connected layer is four layers; in step 1), the obtained initial parameters are the parameters of the fully connected layer; in step 2), only the parameters of the weight gate layer are updated when the point screening model is retrained; the L1 norm of the weight gate layer is added to obtain a sparse result, and the L2 norm of the parameters of the fully connected layer is added to limit the complexity of the model.

2. The DNA methylation-based genetic point screening method according to claim 1, wherein, The gene point is a schizophrenia gene point.

3. The DNA methylation-based genetic point screening method according to claim 1, wherein, The method further comprises: classifying the obtained feature selection result.

4. The DNA methylation-based genetic locus screening method according to claim 3, wherein The classification processing is: inputting the result of feature selection into a linear SVM classifier for classification after compression encoding.

Images