A deep learning-based method for identifying resistance peptides

Abstract

The present invention is applicable to the field of bioinformatics, and provides a method for identifying resistance polypeptides based on deep learning. The present invention focuses on identifying seven biological resistance peptides, namely AMP, ACP, ADP, AHP, AIP, AAP and AOP, and solves the problems of expensive, laborious and time-consuming when identifying resistance polypeptides by traditional methods. The method can more comprehensively describe the characteristics of resistance polypeptides and achieve more efficient and accurate identification. When compared with the five most advanced methods on the same test set, the AP-FD model of the present invention increased the sensitivity by 11.2-23.9%, the accuracy by 0.4-2.5%, the F1 value by 6.1-16%, the MCC by 5.3-16%, the AUC by 1.6-4.7%, and the AP value by 8.7-19.5%. The AUROC and APROC curves further confirmed that the AP-FD model is superior to other comparative methods.

Description

Technical Field

[0001] The present invention belongs to the field of bioinformatics, and in particular relates to a resistance polypeptide identification method based on deep learning. Background Art

[0002] Resistance peptides are a class of small molecule peptides with multiple resistance functions, which are widely found in animals, plants and food. Since resistance peptides play an important role in regulating various functions of the body, peptide drugs also have significant therapeutic effects, short half-life and low toxicity. In recent years, more and more resistance peptides with different functions have been discovered. Experimental data show that many biological resistance peptides have multiple functions. For example, antimicrobial peptides (AMPs) can protect the host from pathogens, and some AMPs also have other functions such as antibacterial, antifungal, antiviral and anticancer. Antiangiogenic peptides (AAPs) are used in the field of cancer treatment due to their function of reducing or inhibiting angiogenesis, and also play an important role in the treatment of other diseases that rely on angiogenesis. In addition, antioxidant peptides (AOPs) extracted from natural proteins may have other biological resistance properties such as cholesterol reduction or antibacterial in addition to their own antioxidant function. Accurately predicting the functional resistance of biological resistance peptides has multiple meanings.

[0003] Given the wide application prospects of resistance peptides, methods for large-scale identification of their functions are particularly important. Traditional experimental methods are difficult to meet the needs, prompting computational methods to become a research hotspot. In recent years, the following methods have been proposed: AMP-BERT adopts the bidirectional encoder architecture of BERT. At the same time, methods combining the Transformer neural network architecture with natural language processing knowledge have also been used to identify and predict the functions of AMPs. The iAMPCN framework can predict AMPs and their functional resistance by integrating four sequence features. For the prediction of ACPs, data augmentation techniques (such as ACP-DA) are effectively applied to solve the problem of insufficient sample size while improving the prediction performance of the model. Subsequently, the prediction accuracy of ACPs was further improved by optimizing features. For the prediction of AAPs, researchers extracted useful descriptive information from different aspects such as PseAAC, atomic spectroscopy, and physical and chemical spectroscopy to construct models (such as AntAngioCOOL). In the application of deep learning methods, the deep learning method MLBP based on convolutional neural networks (CNN) and gated recurrent units (GRU) has become a research hotspot. Compared with MLBP, the multi-resistance prediction method MPMABP was improved by constructing a residual network to ensure that peptide sequence information is not lost and replacing GRU with LSTM. However, MPMABP used the same dataset as MLBP, in which the types of biological resistance peptides were not increased, and the data imbalance problem was not solved.

[0004] The identification of biological resistance peptides with multiple functions is a multi-label classification problem. Therefore, multi-label classification methods from the fields of text, image and bioinformatics in the past few decades can also be used to identify biological resistance peptides. For example, CLR, CPNL, RBRL, MLDF and RAKEL can be used for reference. However, among the various methods mentioned above, the multi-label classification methods that can be applied to multiple tasks have limited prediction capabilities when used alone for the multifunctional identification of biological resistance peptides. The accuracy of recognition is also affected by the zero padding of the peptide sequence, and it cannot provide comprehensive and accurate functional identification. MLBP and MPMABP use peptide sequences as the only input information. When faced with more categories of peptide function identification, their accuracy drops significantly. For this reason, a resistance peptide identification method based on deep learning is proposed. Summary of the invention

[0005] The purpose of the present invention is to provide a method for identifying resistance polypeptides based on deep learning, aiming to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for identifying resistance polypeptides based on deep learning, comprising the following steps:

[0008] Step 1: Data preparation; the specific steps are as follows:

[0009] Step 11, data collection: collect peptide sequence data from the database, including seven categories: AMP, ACP, ADP, AHP, AIP, AAP and AOP, to form a data set; randomly divide the data set into a training set and a test set;

[0010] Step 12: Data balancing: Perform data enhancement on the categories with less data in the training set, and keep the test set unchanged;

[0011] Step 2, sequence encoding: use three different encoding methods, ESM-2, Prot-T5 and AAindex1, to convert the peptide sequence;

[0012] Step 3: AP-FD model construction; the AP-FD model is a deep learning model based on dual channels. The AP-FD model is trained using training set data, and the performance of the AP-FD model is evaluated using the test set. The processing flow of the AP-FD model is as follows:

[0013] The first channel, time series information extraction: merge the peptide sequences encoded by ESM-2, Prot-T5 and AAindex1, and use the bidirectional long short-term memory network (BiLSTM) to learn the time series information of the merged peptide sequences, denoted as S1;

[0014] The second channel, interaction information extraction: The features obtained by ESM-2, Prot-T5 and AAindex1 are combined in pairs to generate joint matrices, which are (AAindex1, Prot-T5), (AAindex1, ESM-2) and (Prot-T5, ESM-2) respectively; a two-dimensional convolutional neural network is used to learn the joint matrix information to obtain the complementary enhancement information between the features, which is recorded as S2;

[0015] Feature classification and function prediction: The information of S1 and S2 is combined, and the combined information is further learned using a fully connected layer. The function of the antagonistic peptide is predicted through the output of the fully connected layer.

[0016] Furthermore, in step 12, the specific operation of performing the data enhancement step on the category with less data in the training set is: generating a new sequence by randomly replacing an amino acid site in a sequence position, and the category of the new sequence is consistent with that of the original sequence.

[0017] Furthermore, in step 2, for a polypeptide sequence of length L, AAindex1 represents a polypeptide sequence as L ×566-dimensional two-dimensional matrix, Prot-T5 represents a peptide sequence as L ×1024-dimensional two-dimensional matrix, ESM-2 represents multiple sequences as L A two-dimensional matrix of dimensions ×1280.

[0018] Furthermore, the output of the AP-FD model is seven categories, namely AMP, ACP, ADP, AHP, AIP, AAP and AOP, each of which is a binary classification, representing resistant polypeptides and non-resistant polypeptides, respectively.

[0019] Furthermore, in step 11, 20% of the samples in the data set are used as a test set, and 80% of the samples are used as a training set.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention provides a deep learning-based resistance peptide identification method, which focuses on identifying seven key biological resistance peptides, including AMP, ACP, ADP, AHP, AIP, AAP and AOP, and aims to solve the problems of expensive, laborious and time-consuming when identifying resistance peptides by traditional methods. The method can more comprehensively describe the characteristics of resistance peptides, thereby achieving more efficient and accurate identification. When compared with the five most advanced methods on the same test set, the AP-FD model of the present invention showed significant performance improvement. Specifically, the AP-FD model increased the sensitivity by 11.2-23.9%, the accuracy by 0.4-2.5%, the F1 value by 6.1-16%, the MCC by 5.3-16%, the AUC by 1.6-4.7%, and the AP value by 8.7-19.5%. The AUROC and APROC curves further confirmed that the AP-FD model is superior to other comparative methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of the present invention.

[0023] Figure 2 Comparison of AUROC and PRROC between the AP-FD model in the present invention and other multi-label classification methods. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0025] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0026] like Figure 1 As shown, a method for identifying resistance polypeptides based on deep learning is provided in one embodiment of the present invention, comprising the following steps:

[0027] Step 1: Data preparation; the specific steps are as follows:

[0028] Step 11, data collection: In order to obtain more resistance peptide data, we used the nine biological resistance peptides related to humans retrieved from the Google Scholar engine in 2020 provided by Tang et al., and selected seven biological resistance peptides, AMP, ACP, ADP, AHP, AIP, AAP and AOP, with more than 100 samples. Among them, the five biological resistance peptides AMP, ACP, ADP, AHP, and AIP used the same experimental data set as MLBP and MPMABP, the AAP data set used the same experimental data set as AntAngioCOOL, and the AOP used the same data set as AnOxPePred. These data were screened by the clustering tool CD-HIT, and the threshold was set to 0.9. Because there are a large number of sequences with 2-4 amino acids in the AOP data set, we processed all the data uniformly and selected peptide sequences with no less than 5 amino acids. Finally, we obtained 2409 AMPs, 646 ACPs, 514 ADPs, 868 AHPs, 1678 AIPs, 134 AAPs, and 318 AOPs. We randomly selected 20% of the samples from the dataset as the test set and used the remaining 80% of the samples as the training set. The details of the training set and test set are shown in Table 1.

[0029]

[0030] Step 12, data balance: Due to the imbalance in the number of data between different categories, we adopted a data enhancement method to increase the categories with too little data. Randomly replace a certain amino acid character in the peptide sequence with other amino acid characters. Since a certain amino acid character in the peptide sequence is only replaced by a new amino acid character each time, it will not affect the category of the original sequence. Therefore, the processed sequence and the original sequence belong to the same category and are regarded as a new sequence, which solves the problem of imbalance in the number of data. It is worth noting that the data enhancement operation is only performed on the training set, and the test set remains unchanged. The added data will be placed after the original data according to the category. Finally, the results shown in Table 2 are obtained, which do not contain repeated sequences.

[0031]

[0032] Step 2, sequence encoding: use three different encoding methods, ESM-2, Prot-T5 and AAindex1, to convert the peptide sequence;

[0033] (1) Peptide sequence encoding based on amino acid properties: The indicators in the AAindex1 database can be applied to the structural feature analysis of amino acids. These indicators provide quantitative information about the various properties and characteristics of amino acids in terms of structure, and can be used to reveal the interactions, stability and structure-function relationships between amino acids. First, some indicators in the AAindex1 database can describe the chemical properties of amino acids, such as the polarity, hydrophobicity and charge of the side chain. These indicators can be used to analyze the position and interaction of amino acids in protein structures. For example, polar amino acids are exposed on the surface, while hydrophobic amino acids are usually buried in the internal core region. Secondly, some indicators in the AAindex1 database also provide information about the physical properties of amino acids, such as size, volume and solubility. These indicators can be used to study the spatial arrangement and stereo configuration of amino acids in the three-dimensional structure of proteins. In addition, some indicators in the AAindex1 database also include information related to structural analysis, such as secondary structure preference, accessibility of amino acid side chains and structural stability. These indicators can be used to predict and analyze the secondary structure, folding state and stability of proteins, as well as to identify structural domains and structural motifs. AAindex1 provides 566 representation information for each amino acid. For a polypeptide of length L, it can be represented as Dimensional matrix.

[0034] (2) Feature encoding based on protein pre-training model: In terms of protein structure prediction and function research, Prot-T5 and ESM-2 are two different methods, which represent the application of two deep learning methods in protein research. Prot-T5 is a deep learning model used to predict the functional and structural properties of protein sequences. It is an application of the T5 model (Text-to-Text Transfer Transformer) in the protein field. The T5 model is a pre-trained language processing model based on the Transformer architecture, which was originally used for natural language processing tasks. Prot-T5 is based on the T5 model, which is a sequence-to-sequence model based on Transformer. It consists of an encoder and a decoder that can process input sequences and generate output sequences. This architecture performs well in processing long sequences and capturing dependencies in sequences. The Prot-T5 model uses the UniRef50 dataset for pre-training. UniRef50 is a protein sequence database that reduces the size of the database by clustering similar protein sequences while retaining enough information to represent the diversity of proteins. During the pre-training process, the model learns the grammatical and semantic information of protein sequences, so that it can predict the functional and structural properties of proteins. The Prot-T5 model can be used for a variety of protein-related tasks, such as protein function prediction, structure prediction, interaction prediction, etc. By fine-tuning the pre-trained model on a specific task, the performance and adaptability of the model can be further improved. The Prot-T5 model converts protein sequences into high-dimensional vector representations that contain protein sequence information and potential biological function information. This representation can be used for downstream tasks such as classification, regression, or generation tasks. Prot-T5 can represent each amino acid as a 1024-dimensional vector. For a polypeptide sequence of length L, Prot-T5 can represent the sequence as Through its high-dimensional vector representation, we can deeply explore the syntax and semantic information of the peptide sequence and provide richer representation information for the prediction of the resistance peptide function.

[0035] Evolutionary Scale Modeling version 2 (ESM-2) is another advanced protein structure prediction model developed by the Facebook AI Research (FAIR) team. The ESM-2 model uses large-scale protein sequence data to train deep neural networks to predict the tertiary structure of proteins. The ESM-2 model is based on the Transformer architecture, a powerful sequence model that can capture long-range sequence dependencies. The Transformer model has achieved great success in the field of natural language processing, and ESM-2 applies it to the processing of protein sequences to predict protein structures. The ESM-2 model infers its structural properties by analyzing the evolutionary information of protein sequences. Conservation patterns in protein sequences can provide clues about their structure and function. ESM-2 uses these patterns to learn the relationship between protein sequence and structure. The ESM-2 model is trained on large-scale protein sequence datasets, which enables it to learn rich protein structure and function information. The training data usually comes from multiple databases, such as UniRef, Pfam, etc., containing hundreds of millions of protein sequences. The ESM-2 model is trained to predict the structural properties of proteins, such as the distance matrix or angle distribution between amino acids. These structural properties can be further used to generate a three-dimensional structural model of the protein. The output of the middle layer of ESM-2 can be used as a representation of the protein sequence. The output dimension of the middle layer of ESM-2 is , where L represents the length of the protein sequence and 1280 represents the dimension of each amino acid representation. Therefore, for a polypeptide of length L, it can be represented as The matrix of .

[0036] In order to verify the synergistic effect of the three types of coding features, AAindex1, Prot-T5 and ESM-2, the synergistic relationship of AAindex1, Prot-T5 and ESM-2 will be calculated, and the prediction ability of AAindex1, Prot-T5 and ESM-2 coding features on the function of antagonistic peptides will be judged by comparing the performance of the feature combination of AAindex1 (A), Prot-T5 (P), ESM-2 (E), AAindex1+Prot-T5 (A+P), AAindex1+ESM-2 (A+E), Prot-T5+ESM-2 (P+E) and AAindex1+Prot-T5+ESM-2 (A+P+E). Table 3 shows the combination results of the three types of coding features.

[0037]

[0038] It can be found from Table 3 that A+P+E improves sensitivity by 0.4-9%, accuracy by 0-0.6%, F1 value by 0.9-5.5%, MCC by 0.7-4.8%, and AP by 1-6.7% in the following seven evaluation indicators: specificity, sensitivity, accuracy, F1 value, MCC, AUC, and AP. Therefore, the combination of AAindex1, Prot-T5, and ESM-2 shows better evaluation indicators on the test set than other features and feature combinations, indicating that the combination has a positive effect on predicting the function of resistance polypeptides.

[0039] Step 3: AP-FD model construction; the AP-FD model is a deep learning model based on dual channels. The training set data is used to train the AP-FD model, and the test set is used to evaluate the performance of the AP-FD model. The processing flow of the AP-FD model is as follows:

[0040] The first channel, time series information extraction: merge the peptide sequences encoded by ESM-2, Prot-T5 and AAindex1, and use the bidirectional long short-term memory network (BiLSTM) to learn the time series information of the merged peptide sequences, denoted as S1;

[0041] The second channel, interaction information extraction: The features obtained by ESM-2, Prot-T5 and AAindex1 are combined in pairs to generate joint matrices, which are (AAindex1, Prot-T5), (AAindex1, ESM-2) and (Prot-T5, ESM-2) respectively; a two-dimensional convolutional neural network is used to learn the joint matrix information to obtain the complementary enhancement information between the features, which is recorded as S2;

[0042] Feature classification and function prediction: The information of S1 and S2 is merged, and the fully connected layer is used to further learn the merged information. The function of the antagonistic peptide is predicted through the output of the fully connected layer.

[0043] The output of the AP-FD model is seven categories, namely AMP, ACP, ADP, AHP, AIP, AAP and AOP, each of which is a binary classification, representing resistant polypeptides and non-resistant polypeptides, respectively.

[0044] Example 1, construction of AP-FD (APFinder) model;

[0045] First, the peptide sequence is represented as a matrix using the AAindex1, Prot-T5 and ESM-2 models. , and The length of resistance peptides is generally 5-40 amino acids, and most of them are less than 30 amino acids. For the convenience of calculation, we choose the peptide length to be 32, where 32 is the 95% quantile of the peptide length in the data set. Therefore, AAindex1 can represent a sequence as A two-dimensional matrix of dimension , Prot-T5 can represent a polypeptide sequence as A two-dimensional matrix of dimension , ESM-2 can represent multiple sequences as A two-dimensional matrix of dimension .

[0046] The APFinder deep learning model is divided into two paths. One path is the input matrix combined with AAindex1, Prot-T5 and ESM-2 encoding methods, expressed as , whose dimension is 32×2870, uses Bi-LSTM to learn the time series information of the sequence and obtain the output result ; The other is a two-dimensional convolutional neural network, learning (AAindex1, Prot-T5) is expressed as , (AAindex1, ESM-2) is expressed as and (Prot-T5, ESM-2) is represented as The joint matrix information of the features is obtained to obtain the complementary enhancement features between the features and obtain the output results .

[0047] and , and as well as and The merging process of is called feature alignment. The process of feature alignment is: through the principal component PCA dimensionality reduction method, Dimensionality reduction to Dimension, expressed as ; Dimensionality reduction to Dimension, expressed as ; Dimensionality reduction to Dimension, expressed as By merging and , and , and Construct a 32×32×6 input matrix, learn the fused features through a convolutional neural network, and output the network learning results .

[0048] By combining the information of S1 and S2 and using a fully connected network to further learn the combined information of S1 and S2, the function of the resistance peptide is predicted. The output of the model is divided into 7 categories. Each category is also divided into two categories, represented by resistance peptides and non-resistance peptides. The loss function of the network is defined as ,in is the label of sample i, is the predicted value. The weights of the two-way neural network are learned through the Adma optimizer, and the AP-FD model is finally constructed.

[0049] Example 2: In order to verify the predictive ability of the AP-FD model, the AP-FD model was compared with other multi-label classification methods. So far, MLBP and MPMABP are the latest methods for classifying multifunctional marker biological resistance peptides. MLBP uses a deep learning method that combines CNN and Bi-GRU; MPMABP builds a residual network based on MLBP and replaces GRU with LSTM. At the same time, we also selected other multi-label algorithms that can predict resistance peptides, such as multi-label deep forest (MLDF) using measurement-aware feature reuse and measurement-aware layer growth; calibrated label ranking (CLR) combines pairwise preference learning and traditional related classification techniques; random k label set (RAKEL), which considers label correlation by using a single label classifier. We use five-fold stratified cross-validation on the training set. In order to avoid overfitting when performing five-fold cross-validation experiments on the extended training set, which may lead to deviations in the experimental results, we chose to conduct experiments on the original test set. The comparison results are shown in Table 4.

[0050]

[0051] It can be found from Table 4 that in terms of the following seven evaluation indicators, specificity, sensitivity, accuracy, F1 value, MCC, AUC and AP, AP-FD is slightly inferior to other methods in specificity, but it is significantly superior to other comparison methods in sensitivity, accuracy, F1 value, MCC, AUC and AP. Specifically, AP-FD increased the sensitivity by 11.2-23.9%, the accuracy by 0.4-2.5%, the F1 value by 6.1-16%, the MCC by 5.3-16%, the AUC by 1.6-4.7%, and the AP value by 8.7-19.5%. The results show that AP-FD performs well in identifying the function of resistance polypeptides. In addition, in order to further demonstrate the differences in the performance of each method, Figure 2 The AUROC and APROC curves of each method are shown, which further confirms that AP-FD is indeed superior to other compared methods.

[0052] Conclusion: This paper proposes a new method for identifying resistance polypeptides. In order to train AP-FD, we searched and obtained 2409 AMP, 646 ACP, 514 ADP, 868 AHP, 1678 AIP, 134 AAP and 318 AOP polypeptide sequences from existing research databases. In order to balance the resistance polypeptide sequence data of each category, the amino acid site in a sequence position was randomly replaced, which not only ensured the function of the original resistance polypeptide of each category, but also balanced the resistance polypeptide sequence data of each category. The polypeptide sequence was converted into a two-dimensional matrix by AAindex1, Prot-T5 and ESM-2 encoding. The combination experiment of encoding features showed that the combination encoding method of AAindex1, Prot-T5 and ESM-2 was helpful for predicting the function of resistance polypeptides. By constructing a deep learning model, the time series information of the sequence and the complementary enhancement information between the features were learned respectively. Compared with other comparison methods, AP-FD performed better in the test set. Compared with MLBP, MPMABP, RAKEL, MLDF and CLR methods, AP-FD improved the sensitivity by 11.2-23.9%, the accuracy by 0.4-2.5%, the F1 value by 6.1-16%, the MCC by 5.3-16%, the AUC by 1.6-4.7%, and the AP value by 8.7-19.5%. The AUROC and APROC curves show that AP-FD is indeed superior to other comparison methods.

[0053] The above are only preferred embodiments of the present invention. It should be pointed out that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention. These should also be regarded as the protection scope of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A method for identifying resistance polypeptides based on deep learning, characterized in that: The following steps are involved: Step 1: Data preparation; the specific steps are as follows: Step 11, data collection: collect peptide sequence data from the database, including seven categories: AMP, ACP, ADP, AHP, AIP, AAP and AOP, to form a data set; randomly divide the data set into a training set and a test set; Step 12: Data balancing: Perform data enhancement on the categories with less data in the training set, and keep the test set unchanged; Step 2, sequence encoding: use three different encoding methods, ESM-2, Prot-T5 and AAindex1, to convert the peptide sequence; Step 3: AP-FD model construction; the AP-FD model is a deep learning model based on dual channels. The training set data is used to train the AP-FD model, and the test set is used to evaluate the performance of the AP-FD model. The processing flow of the AP-FD model is as follows: The first channel, time series information extraction: merge the peptide sequences encoded by ESM-2, Prot-T5 and AAindex1, and use the BiLSTM network to learn the time series information of the merged peptide sequences, denoted as S1; The second channel, interaction information extraction: The features obtained by ESM-2, Prot-T5 and AAindex1 are combined in pairs to generate joint matrices, which are (AAindex1, Prot-T5), (AAindex1, ESM-2) and (Prot-T5, ESM-2) respectively; a two-dimensional convolutional neural network is used to learn the joint matrix information to obtain the complementary enhancement information between the features, which is recorded as S2; Feature classification and function prediction: The information of S1 and S2 is merged, and the fully connected layer is used to further learn the merged information. The function of the antagonistic peptide is predicted through the output of the fully connected layer.

2. The method for identifying resistance polypeptides based on deep learning according to claim 1, characterized in that: In step 12, the specific operation of performing data enhancement on the category with less data in the training set is: generating a new sequence by randomly replacing an amino acid site in a sequence position, and the category of the new sequence is consistent with that of the original sequence.

3. The method for identifying resistance polypeptides based on deep learning according to claim 1, characterized in that: In step 2, for a polypeptide sequence of length L, AAindex1 represents a polypeptide sequence as L ×566-dimensional two-dimensional matrix, Prot-T5 represents a peptide sequence as L ×1024-dimensional two-dimensional matrix, ESM-2 represents multiple sequences as L A two-dimensional matrix of dimensions ×1280.

4. The method for identifying resistance polypeptides based on deep learning according to claim 1, characterized in that: The output of the AP-FD model is seven categories, namely AMP, ACP, ADP, AHP, AIP, AAP and AOP, each of which is a binary classification, representing resistant polypeptides and non-resistant polypeptides, respectively.

5. The method for identifying resistance polypeptides based on deep learning according to claim 1, characterized in that: In step 11, 20% of the samples in the data set are used as a test set, and 80% of the samples are used as a training set.

Images