Small sample data cancer typing method based on transfer learning

Abstract

The invention discloses a small sample data cancer typing method based on transfer learning, and belongs to the field of life health omics data analysis. The method comprises the steps of firstly performing feature selection and dimension reduction processing on to-be-integrated disease omics data, then integrating multiple pieces of disease omics data to construct a disease big data set, training a deep learning base model based on the big data set, and finally applying the base model to disease omics small sample data through one-step or multi-step migration to obtain a final disease type. According to the method, the characteristics of high-dimensional small samples of disease omics data are considered, a disease big data set is constructed, the deep neural network and transfer learning are applied to omics data analysis, and guidance is provided for application of disease diagnosis typing and a deep learning method to omics data.

Description

Technical Field

[0001] The present invention belongs to the technical field of life and health omics data analysis, and relates to a small sample data cancer typing method based on transfer learning. Background Art

[0002] With the development of bio-omics experimental technology, different bio-omics (genomics, transcriptomics, proteomics, metabolomics, etc.) data are coming, and different bio-omics can reflect different physiological and pathological states. Among them, genomics reflects what can happen, transcriptomics reflects what will happen, proteomics points out what depends on it, and metabolomics reflects what has happened. Therefore, the mining and analysis of bio-omics data can realize the typing of complex diseases such as cancer, clarify the affected functions and pathways, and provide support for the early diagnosis and treatment of diseases.

[0003] With the continuous development of deep learning, more and more researchers have begun to apply deep learning models to the research in the field of life and health, such as the application of recurrent neural network (RNN) models to electronic medical record data analysis, and convolutional neural network (CNN) models to medical CT image data analysis. However, unlike medical text data and medical image data, bio-omics data generally has the characteristics of "high dimensionality, small samples, and imbalanced categories", that is, the number of samples contained in omics data is small, and the dimension of the features is much higher than the number of samples. At the same time, the distribution of disease samples and normal samples is unbalanced, which increases the difficulty of analyzing bio-omics data. The lack of training data and the dimensionality disaster are not conducive to the training of deep neural networks. Therefore, it is necessary to apply a suitable data enhancement method to increase the number of training samples and improve the effect of deep model training.

[0004] In order to effectively solve the problem of small sample size, many researchers use enhancement technology to increase the sample size and improve the performance of subsequent classifiers. Generative adversarial networks are the main data enhancement technology. Its core idea is to form an adversarial relationship between two neural networks, the discriminator and the generator, and continuously iterate and optimize to generate high-quality new samples that are close to the original samples. Although the use of generative adversarial networks can increase the number of samples, the interaction between molecules is ignored in the process of generating new samples, and the reliability of new data is poor.

[0005] When dealing with small sample data sets, in addition to using data augmentation technology to expand the data set, transfer learning technology can also be used. Transfer learning relaxes the assumption that training data and test data are independent and identically distributed. Its core idea is to reduce the data requirements of new tasks, improve model performance, or accelerate model convergence by transferring existing knowledge from the source domain to the target domain. However, in existing transfer learning technologies, if deep models such as DNN and ResNet (Residual Network) are used as base models, the demand for training data is still large, and for single-category bio-omics data, it cannot effectively meet the training requirements. At the same time, because the source domain and the target domain have certain similarities, the training method of transfer still needs to be improved.

[0006] Therefore, the present invention proposes a small sample classification algorithm for cancer omics based on transfer learning (SDCM_TL); the SDCM_TL method integrates multiple cancer omics data, constructs cancer big data, trains a deep neural network base model, and migrates it to a small sample of cancer omics for cancer classification. Summary of the invention

[0007] The purpose of the present invention is to address the problem of insufficient model training caused by deep learning models in processing small sample disease omics data, propose a disease omics data integration and migration model, construct disease big data, which can improve the training accuracy of the model to a certain extent, and use migration learning to perform effective data analysis and mining on small sample omics data. The present invention combines the actual characteristics of transcriptomics data with many variables and small sample size under current high-throughput technology and cannot train deep learning models, integrates public transcriptomics data into large data sets, trains deep learning models, and migrates the model to small sample cancer data sets for cancer typing. The model is suitable for the analysis and research of small sample bio-omics data, and can be used in the fields of proteomics data analysis, genomics data analysis, metabolomics data analysis, and precision medicine.

[0008] The core technology of this method is to first perform feature selection and dimensionality reduction on the disease omics data to be integrated, then integrate multiple cancer omics data to construct a large disease data set, and then train the deep learning base model based on this large data set. Due to the heterogeneity between different disease omics data, different migration methods are used for different small disease data sets. If the disease to be analyzed is in the constructed disease big data set, then a one-step migration method is used to fine-tune the base model and classify the disease; if the disease to be analyzed is not in the constructed disease big data set, then a two-step migration method is used to fine-tune the base model and classify the disease.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0010] A small sample data cancer typing method based on transfer learning, comprising the following steps:

[0011] Construct a disease omics dataset consisting of data from different disease categories;

[0012] Preprocessing the data in the disease omics dataset;

[0013] Perform feature selection and dimensionality reduction on the preprocessed disease omics data;

[0014] Integrate the reduced-dimensional data to construct a large disease omics dataset;

[0015] A general disease model is obtained by training based on the disease omics big data set;

[0016] The disease general model is migrated to the disease omics small sample data in one or more steps to obtain the disease typing.

[0017] Furthermore, the disease omics data set is composed of data sets of different disease categories, each disease category data set is composed of several samples, each sample is composed of several feature (matters reflecting the performance or nature of an object in a certain aspect) data; the data source includes The Cancer Genome Atlas (TCGA); The Cancer Genome Atlas (TCGA) is a database containing multiple cancer types created by the Cancer Genome Project. The present invention collects and organizes the omics data in the TCGA cancer database for subsequent construction and analysis of cancer big data.

[0018] Furthermore, the data in the disease omics dataset is preprocessed, including:

[0019] First, missing value processing is performed. The specific method is as follows:

[0020] If the number of missing feature values ​​(the value taken by the sample on this feature) of a sample in a certain disease category exceeds 20% of the total number of features of all samples in this disease category, the sample is deleted; otherwise, the average value of the feature values ​​of all remaining samples in this disease category is used to fill the missing value.

[0021] Then, the data after missing value processing is standardized. The specific method is as follows:

[0022] The Z-Score method is used to standardize the data, and the calculation formula is:

[0023]

[0024] Among them, Z is the standardized value of the feature corresponding to each sample in each disease category, X is the original value of the feature corresponding to each sample in each disease category, μ is the mean of the feature on all samples in the disease omics dataset, and σ is the standard deviation of the feature on all samples in the disease omics dataset.

[0025] Furthermore, feature selection and dimensionality reduction are performed on the preprocessed disease omics data. The specific process is as follows:

[0026] In order to select disease-related features, simplify the number of data features, reduce the time complexity and space complexity of deep learning model training, and obtain disease-specific molecules, symmetric uncertainty (SU) is used to select potential markers related to the disease. That is, the larger the symmetric uncertainty value between the feature and the disease category, the stronger the predictive ability of the feature for the disease; the SU calculation formula is:

[0027]

[0028] Among them, F represents a feature in the disease omics dataset; C represents the disease category; H(F) and H(C) represent the information entropy of feature F and disease category C respectively; MI(F,C) represents the mutual information between feature F and disease category C; SU(F,C) is the symmetric uncertainty value of feature F and disease category C, which represents the normalized mutual information. Its value range is [0,1], reflecting the dependency between the feature and the disease category. The larger the value, the stronger the correlation between feature F and disease category C.

[0029] The symmetric uncertainty value of each feature F and disease category C is compared with a pre-set threshold, and the corresponding features whose symmetric uncertainty values ​​exceed the threshold are screened out.

[0030] Furthermore, the process of constructing a disease omics big data set is as follows:

[0031] The data after dimensionality reduction are integrated and processed to obtain small data sets of each disease category, and the small data sets of each disease category are merged to obtain the disease omics big data set; among them, the features in the disease omics big data set are the union of the features contained in each small data set, and the samples in the disease omics big data set are all the samples of the small data sets. At the same time, a sample label is assigned to each sample in the disease omics big data set, and the disease sample is assigned the label "1", and the normal sample is assigned the label "0".

[0032] Furthermore, a deep neural network model is trained based on the disease omics big data set, wherein the deep neural network model includes four hidden layers and one output layer, and the model parameters are optimized through iterative learning to obtain a general disease model as a base model of the migration model.

[0033] Furthermore, in view of the characteristics of small sample data of diseases, when the category d* of the small sample data of the target disease belongs to the known disease category set D in the disease omics big data set, a one-step migration method is adopted; when the category d* of the small sample data of the target disease does not belong to the existing disease category set D, a two-step migration method is adopted.

[0034] Furthermore, when the category d* of the small sample data of the target disease belongs to the known disease category set D in the disease omics big data set, a one-step migration method is adopted to migrate the base model using the data set of the target disease category d*, lock the first two hidden layers of the base model, only adjust the last two hidden layers of the base model, and add a new output layer to replace the original output layer to obtain a disease-specific model, and use the disease-specific model to classify the small sample data of the target disease.

[0035] Furthermore, when d* does not belong to the existing disease category set D, a two-step migration method is used for fine-tuning, as follows: first, the base model is migrated using an extended large data set containing the target disease category d*. The specific method is to lock the first two hidden layers of the base model, only adjust the last two hidden layers of the base model, and add a new output layer to replace the original output layer, so as to obtain a general model adapted to the extended data; then the second step of migration is performed, that is, the general model is used to perform further migration and fine-tuning on the small sample data of the target disease corresponding to d*, by freezing the first three hidden layers of the general model, only adjusting the last hidden layer of the general model, and also adding a new output layer to replace the original output layer of the general model, so as to better adapt to the characteristics of the small sample disease d*; after such two-step migration training, a disease-specific model for d* is finally obtained for disease typing tasks.

[0036] Beneficial effects of the present invention: The present invention is used for the construction of a deep model of a small data set and cancer typing. The method integrates multiple disease transcriptomics data, constructs cancer omics big data, trains a deep neural network model based on big data, and migrates the trained model to a small sample cancer data set through one or more steps for small sample cancer typing. The method takes into account the characteristics of high-dimensional small samples of disease omics data, constructs a large disease data set, and applies deep neural networks and transfer learning to omics data analysis, providing guidance for disease diagnosis typing and the application of deep learning methods on omics data. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is the overall flow chart of the method of the present invention. DETAILED DESCRIPTION

[0038] The embodiments of the present invention are implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments.

[0039] In order to solve the problem that the sample size of disease omics is small and the deep model cannot be trained, the present invention proposes a small data cancer sample classification method based on transfer learning. The method first collects multiple disease omics data and performs preprocessing, calculates the symmetric uncertainty of features and disease categories, selects disease-related biological molecules, and constructs a large disease omics data set; then, the deep neural network base model is trained using the disease omics large data set to obtain a general disease model as the base model; finally, the base model is migrated in one or more steps and applied to the small sample data of disease omics to obtain the final disease classification. Taking the human cancer transcriptome data set as an example, the execution process is briefly described. The specific steps are as follows:

[0040] Step 1: Collect and organize cancer omics data.

[0041] The present invention collected and sorted out thirteen cancer transcriptomics mRNA expression datasets from the TCGA database. The diseases involved in these datasets include bladder cancer, breast cancer, colorectal cancer, endometrial cancer, head and neck squamous cell carcinoma, renal clear cell carcinoma, renal papillary cell carcinoma, liver cancer, lung cancer, lung squamous cell carcinoma, prostate cancer, gastric cancer and thyroid cancer. All of them are binary classification problems, with a feature dimension of 20531 and a sample number variation range of [194,830]. Table 1 shows the detailed information of the dataset.

[0042] Table 1 TCGA cancer transcriptomics data

[0043]

[0044] For these thirteen transcriptomics datasets from different cancers: the number of samples in the dataset is far less than the number of features, each sample is divided into a cancer sample group and a normal sample group according to its physiological and pathological state, and the two categories are unevenly distributed in different diseases. Taking prostate cancer (PRAD) as an example, the dataset contains 491 prostate cancer tissues, 52 normal tissues, and the number of features (genes) is 20,531.

[0045] Step 2: Preprocessing cancer omics data is an important basis for data analysis and modeling, which can improve the accuracy and stability of the model. The preprocessing process includes two parts: missing value filling and Z-Score standardization.

[0046] For the missing value filling logic of cancer omics data, the method adopted is: if the number of missing feature values ​​(the value taken by the sample on the feature) of a sample in a certain disease category exceeds 20% of the total number of features of all samples in the disease category, then delete the sample, otherwise, use the average value of the feature values ​​of all other samples in the disease category to fill the missing value. This strategy can effectively maintain data consistency and minimize information loss.

[0047] Next, the data after missing value processing is Z-Score standardized. Z-Score standardization subtracts the mean value of each feature from the value of the feature and divides it by the standard deviation of the feature, so that the data has a standard normal distribution with a mean of 0 and a standard deviation of 1. This standardization process can eliminate the difference in dimensions between different features, so that the model has a more balanced weight distribution for each feature during the training process, avoiding the impact of model performance due to large differences in the numerical ranges of different features.

[0048] Step three, use feature selection methods to screen the data of different cancers and extract feature subspaces with higher predictive power, thereby helping the model to more accurately identify and distinguish different types of cancer. The present invention uses the Symmetrical Uncertainty (SU) method to calculate the correlation between features and disease categories. This method can effectively identify potential markers that are highly correlated with disease categories.

[0049] After calculating the symmetric uncertainty value between each feature and the disease category, the features are screened according to these values. Assuming the threshold is 0.2, if the symmetric uncertainty value SU(F,C) between feature F and disease category C is ≥0.2, the feature is considered to have a strong correlation with the disease category and is retained for subsequent model training; conversely, if SU(F,C)<0.2, the feature is considered to contribute less to disease prediction and is eliminated. In this way, a feature subspace that is closely related to a specific cancer category is extracted. This step can effectively reduce the data dimension and the training complexity of the deep learning model, while ensuring that the model focuses on features that are important for disease classification.

[0050] Through feature selection, different cancer types will correspond to different feature subspaces, which will be used in subsequent model training to further improve the classification performance and generalization ability of the model.

[0051] Step 4: Integrate multiple small-scale data sets to build a comprehensive large data set containing multiple cancer types. In this process, small data sets from different cancer omics data are unified and integrated to form a more representative and diverse large data set. The samples in each small data set are used as the source of samples in the large data set, and the features of these samples will become the feature components of the large data set.

[0052] To ensure the uniformity and consistency of the data, all samples in the small data sets were relabeled. In the large data set, disease samples were labeled "1", indicating that the sample came from a cancer patient; while normal samples were labeled "0", indicating that the sample came from a healthy individual. This label design ensures that in the subsequent deep learning model training, the model can correctly distinguish cancer samples from normal samples based on the input data.

[0053] Step 5: Use a deep neural network (DNN) model to train the cancer classification task. The architecture of the DNN model has five layers, including four hidden layers and one output layer, with the number of hidden layer nodes being 512, 256, 128, and 64, respectively. The design of the hidden layer gradually reduces the number of neurons so that the model can gradually extract the feature hierarchy in the data and effectively reduce the risk of overfitting. The output layer of the model is used to complete the classification task, and the prediction and classification of the disease is achieved by outputting a binary label (1 for cancer samples and 0 for normal samples).

[0054] In order to optimize the performance of the model, the grid search method was used to tune multiple key parameters of the DNN model. In this process, multiple parameters were systematically tested and compared, with the goal of finding the best parameter combination to improve the prediction accuracy and generalization ability of the model. The parameters involved in parameter optimization include learning rate, activation function, number of training rounds, and training size per round.

[0055] After multiple experiments and comparisons using the grid search method, we determined the parameter combination that provides the best performance. This combination not only efficiently captures data features during training, but also demonstrates strong generalization capabilities on the validation set, resulting in a general disease model with excellent performance. This model will serve as the basis for fine-tuning and transfer learning in subsequent steps, helping to further improve the classification accuracy of different diseases.

[0056] Step 6: Transfer the trained disease general model to the disease small sample omics data to achieve the classification of the target disease. This process uses two different transfer learning methods depending on whether the target disease category d* exists in the known disease category set D in the disease omics big data set.

[0057] First, when the target disease category d* belongs to the disease omics big data set, that is, d*∈D, a one-step migration method is adopted. The method is to lock the first two hidden layers of the base model, only adjust the last two hidden layers, and add a new output layer to replace the original output layer. The last three layers (two hidden layers and one output layer) of the base model are trained to achieve model fine-tuning. In this step, the small sample data corresponding to the target disease category d* is directly used for training. In this way, the model can quickly adapt to the characteristics of a specific disease, thereby obtaining a disease-specific model. During the training process, the model makes full use of the samples of the d* category in the disease omics big data set to improve the classification accuracy and effectiveness of the disease.

[0058] If the target disease category d* does not belong to the existing disease category set D in the disease omics big data set, a two-step migration method is required. First, the first step of transfer learning training of the model is performed using the extended big data set containing the new disease d*. Similar to the one-step migration, the first two hidden layers of the base model are locked, and only the last two hidden layers are adjusted. A new output layer is added to replace the original output layer, and the last three layers (two hidden layers and one output layer) of the base model are trained to achieve model fine-tuning. In this step, the base model is intensively trained on the extended big data set to obtain a general model that can capture a wider range of features in the data and adapt to a variety of disease samples.

[0059] After obtaining the general model, further migration fine-tuning is performed for specific small sample data. This process mainly involves the following operations: locking the first three hidden layers of the general model to keep the features that these layers have learned; then, adding a new output layer to replace the original output layer of the general model, training the last two layers of the general model, and fine-tuning the model to adapt to the characteristics of the target disease d*.

[0060] Next, two small samples (external data) of transcriptomics of cancer (cholangiocarcinoma and renal chromophobe cell carcinoma) are used as examples to demonstrate the effectiveness of the present invention.

[0061] Table 2 Small sample data of cancer transcriptomics

[0062]

[0063] The following table compares the classification performance of the method of the present invention (SDCM_TL, small dataset classification model based on transfer learning) and other classification models commonly used in omics data analysis, including random forest, support vector machine, decision tree, and linear discriminant analysis, on a small sample bio-omics public dataset (i.e., the external data in Table 2) after 50 five-fold cross validations. It can be seen from the results that, in general, the classification performance of the present invention is higher than that of other technologies, proving the effectiveness of the content of the present invention.

[0064] Table 3 Comparison of SDCM_TL with other methods

[0065]

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A small sample data cancer typing method based on transfer learning, characterized in that: include: Construct a disease omics dataset consisting of data from different disease categories; Preprocessing the data in the disease omics dataset; Perform feature selection and dimensionality reduction on the preprocessed disease omics data; Integrate the reduced-dimensional data to construct a large disease omics dataset; A general disease model is obtained by training based on the disease omics big data set; The disease general model is migrated to the disease omics small sample data in one or more steps to obtain the disease typing.

2. According to claim 1, a small sample data cancer typing method based on transfer learning is characterized in that: The disease omics dataset consists of datasets of different disease categories, each disease category dataset consists of a number of samples, and each sample consists of a number of feature data.

3. According to claim 2, a small sample data cancer typing method based on transfer learning is characterized in that: Preprocessing the data in the disease omics dataset includes: Missing value processing: When the number of missing feature values ​​of a sample in a disease category exceeds 20% of the total number of features of all samples in the disease category, the sample is deleted; otherwise, the average value of the feature values ​​of all other samples in the disease category is used to fill the missing value; Data standardization, the specific method is: use the Z-Score method to standardize the data after missing value processing: Among them, Z is the standardized value of the feature corresponding to each sample in each disease category, X is the original value of the feature corresponding to each sample in each disease category, μ is the mean of the feature on all samples in the disease omics dataset, and σ is the standard deviation of the feature on all samples in the disease omics dataset.

4. According to claim 3, a small sample data cancer typing method based on transfer learning is characterized in that: The preprocessed disease omics data are subjected to feature selection and dimensionality reduction. The specific methods are as follows: Symmetric uncertainty was used to select potential disease-related markers, and the calculation formula was: Where F represents the feature in the disease omics dataset; C represents the disease category; H(F) and H(C) represent the information entropy of feature F and disease category C, respectively; MI(F,C) represents the mutual information between feature F and disease category C; SU(F,C) is the symmetric uncertainty value of feature F and disease category C, which represents the normalized mutual information and its value range is [0,1]; The symmetric uncertainty value of each feature F and disease category C is compared with the set threshold, and the corresponding features whose symmetric uncertainty values ​​exceed the threshold are screened out.

5. The method for cancer classification based on small sample data and transfer learning according to claim 4, characterized in that: The process of constructing a disease omics big data set is as follows: integrate the data after dimensionality reduction to obtain small data sets of each disease category, and merge the small data sets of each disease category to obtain a disease omics big data set; the features in the disease omics big data set are the union of the features contained in each small data set, and the samples in the disease omics big data set are all the samples of the small data sets; at the same time, a sample label is assigned to each sample in the disease omics big data set, the disease sample is assigned the label "1", and the normal sample is assigned the label "0".

6. The method for cancer typing based on small sample data and transfer learning according to claim 5, characterized in that: The deep neural network model includes four hidden layers and one output layer. The model parameters are optimized through iterative learning to obtain a general disease model as the base model of the migration model.

7. The method for cancer classification based on small sample data and transfer learning according to claim 6, characterized in that: When the category d* of the small sample data of the target disease belongs to the known disease category set D in the disease omics big data set, a one-step migration method is adopted; when the category d* of the small sample data of the target disease does not belong to the existing disease category set D, a two-step migration method is adopted.

8. The method for cancer typing based on small sample data and transfer learning according to claim 7, characterized in that: The one-step migration method is: locking the first two hidden layers of the base model, only adjusting the last two hidden layers of the base model, and adding a new output layer to replace the original output layer, obtaining a disease-specific model, and using the disease-specific model to classify the small sample data of the target disease.

9. The method for cancer typing based on small sample data and transfer learning according to claim 7, characterized in that: The two-step migration method is as follows: first, use the extended large data set containing the target disease category d* to perform the first step of migration on the base model. The specific method is to lock the first two hidden layers of the base model, only adjust the last two hidden layers of the base model, and add a new output layer to replace the original output layer, so as to obtain a general model adapted to the extended data; then perform the second step of migration, that is, use the general model to further migrate and fine-tune the small sample data of the target disease corresponding to d*: freeze the first three hidden layers of the general model, only adjust the last hidden layer of the general model, and also add a new output layer to replace the original output layer of the general model. After such two-step migration training, a disease-specific model for d* is finally obtained for disease typing tasks.

Images