Classification model construction method and device, electronic device, and storage medium

Abstract

The present application discloses a classification model construction method, the method comprising: obtaining a target object sample set, and extracting target feature information of the target object sample set; determining multiple splitting attributes of the target feature information; wherein the splitting attributes are used to characterize the attributes of the category splitting nodes in the target object sample set; determining the weight values ​​corresponding to the multiple splitting attributes, and obtaining some target splitting attributes with the largest weight values ​​from the multiple splitting attributes; the weight values ​​are used to characterize the category discrimination of the splitting attributes; and constructing a classification model of the target object based on the target feature information and the target splitting attributes. The classification model construction method provided by the present application can improve the training efficiency of the classification model, shorten the training time, and reduce the computational overhead. The present application also discloses a classification model construction device, an electronic device, and a computer-readable storage medium.

Description

Technical Field

[0001] The present application relates to the field of big data processing technology, and in particular to a classification model construction method and device, electronic equipment, and computer-readable storage medium. Background Art

[0002] The study of the relationship between gene expression data and cancer / disease states is crucial in biology and medicine. Comparing gene expression data from diseased tissue with that from normal tissue can deepen our understanding of pathology and identify distinct tissues and disease types.

[0003] Currently, the use of machine learning and deep learning methods to classify gene expression data for cancer and other diseases is attracting industry attention. However, due to the small sample size and high dimensionality of gene expression data, building classification models for gene expression data currently requires a significant amount of training time. Summary of the Invention

[0004] The present application provides a classification model construction method and device, electronic device, and computer-readable storage medium, which can improve the training efficiency of the classification model, shorten the training time, and reduce computing overhead.

[0005] In a first aspect, the present application provides a classification model construction method, the method comprising:

[0006] Acquire a target object sample set, and extract target feature information of the target object sample set;

[0007] Determine a plurality of splitting attributes of the target feature information; wherein the splitting attributes are used to characterize the attributes of the category splitting nodes in the target object sample set;

[0008] Determine weight values ​​corresponding to the multiple split attributes respectively, and obtain some target split attributes with the largest weight values ​​from the multiple split attributes; the weight values ​​are used to represent the category discrimination of the split attributes;

[0009] A classification model of the target object is constructed based on the target feature information and the target splitting attribute.

[0010] In a second aspect, the present application provides a classification model construction device, the device comprising:

[0011] A feature extraction unit, configured to obtain a target object sample set and extract target feature information from the target object sample set;

[0012] a split attribute determination unit, configured to determine a plurality of split attributes of the target feature information; wherein the split attributes are used to characterize the attributes of the category split nodes in the target object sample set;

[0013] a weight value determining unit, configured to determine weight values ​​corresponding to the plurality of split attributes, and to obtain a portion of target split attributes with the largest weight value from the plurality of split attributes; the weight value is used to characterize the class discrimination of the split attribute;

[0014] A processing unit is used to build a classification model of the target object based on the target feature information and the target splitting attribute.

[0015] In a third aspect, the present application provides an electronic device, comprising a processor and a memory for storing a computer program capable of running on the processor;

[0016] Wherein, when the processor is used to run the computer program, it executes the steps of the classification model construction method as described in the first aspect.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the classification model construction method described in the first aspect.

[0018] The classification model construction method and device, electronic device, and computer storage medium provided in the present application obtain a target object sample set, and extract the target feature information of the target object sample set, and determine multiple split attributes of the target feature information; then determine the weight values ​​corresponding to the multiple split attributes, and obtain some target split attributes with the largest weight value from the multiple split attributes; the weight value is used to characterize the category discrimination of the split attribute; based on the target split attribute and the target feature information, a classification model of the target object is constructed. In this way, the target object sample set can be feature extracted to reduce the dimension of the original target object sample set, and at the same time, some split attributes with the largest weight value (i.e., the highest discrimination) are selected from multiple split attributes for model construction, which can greatly reduce the data dimension in the classification model training process and shorten the learning time of the classification model. In this way, by extracting the target feature information after the features are extracted and the selected part of the split attributes for construction of the classification model, the data dimension in the classification model training process can be greatly reduced, thereby improving the training efficiency of the classification model, shortening the training time, and reducing the computational overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a classification model construction method provided in an embodiment of the present application Figure 1 ;

[0020] Figure 2A schematic diagram of the structure of a decision tree provided in an embodiment of the present application Figure 1 ;

[0021] Figure 3 A schematic diagram of a classification model construction method provided in an embodiment of the present application Figure 2 ;

[0022] Figure 4 A schematic diagram of a classification model construction method provided in an embodiment of the present application Figure 3 ;

[0023] Figure 5 A schematic diagram of the structure of a deep forest model provided in this application embodiment Figure 1 ;

[0024] Figure 6 A schematic diagram of the system architecture of a classification model construction method provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of the structural composition of a classification model building device provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present invention.

[0028] In the early days, due to a lack of technical means, the diagnosis of cancer / diseases could only be achieved through clinical laboratory observation, recording, and analysis. With the advent of gene chip technology and gene sequencing, a large amount of gene expression data on cancer / disease tissues has been generated, providing a new approach to cancer / disease research and diagnosis. In practical applications, statistical and machine learning techniques are being applied to classify cancer / disease gene expression data and identify signature genes that may be helpful in cancer / disease treatment and research.

[0029] Currently, many machine learning and deep learning methods have been applied to the classification of gene expression data for cancer / diseases. Cancer / disease classification methods based on machine learning can be roughly divided into four categories:

[0030] (1) Classification method based on similarity

[0031] Similarity-based classification methods primarily determine the category of the gene expression data to be classified based on the similarity (e.g., Euclidean distance) between the gene expression data sample to be classified and the training sample. There are two types of algorithms representing similarity-based classification methods: the K-Nearest Neighbor (KNN) algorithm and the clustering algorithm. The main idea of ​​the KNN-based classification method is to obtain the training sample most similar to the gene expression data to be classified through similarity calculation, and then use the category of the selected training sample as the category of the gene expression data to be classified. In contrast, the main idea of ​​the clustering-based classification method is to use a clustering algorithm to calculate similarity and group similar samples into the same cluster. Then, based on the actual category of other samples in the cluster where the gene expression data to be classified is located, a voting method is used to determine the category of the gene expression data to be classified.

[0032] However, the disadvantage of the similarity-based classification algorithm is that when the dimensions of the gene expression data to be classified and the training samples are large, it takes a lot of time to calculate the similarity between each gene expression data to be classified and each training sample in the training sample set, and the classification efficiency is low.

[0033] (2) Classification method based on maximum margin

[0034] Maximum margin classification methods use linear combinations of sample gene expression data to make classification decisions based on the resulting linear combination. The key is to maximize the classification margin, which is the distance between the classification boundary and the nearest training sample point. Representative algorithms for maximum margin classification include support vector machines (SVMs). These methods involve finding a hyperplane that separates samples from different classes and then making classification decisions based on which side of the hyperplane the gene expression data to be classified lies.

[0035] However, the classification method based on maximum margin works well for linearly separable gene expression data, but its classification effect is not accurate for some nonlinearly separable and noisy data.

[0036] (3) Classification method based on distribution

[0037] Distribution-based classification methods primarily rely on the distribution of the training samples themselves, meaning the gene expression data itself determines the final classification. Representative algorithms in this category include Fisher Linear Discriminant Analysis (FLDA) and the Naive Bayesian algorithm. For example, given the gene expression data to be classified, the Naive Bayesian classification method calculates the probability of each category appearing under the conditions in which the gene expression data appears. The category with the highest probability is considered the category in which the data is classified.

[0038] However, distribution-based classification methods have two problems. First, they assume that genes are independent, while in reality, many gene expression values ​​are correlated. This can overlook many genes that are crucial for classification or biological significance. Second, distribution-based classification algorithms assume that the data follows a Gaussian distribution, but in reality, many gene expression data do not follow this distribution. Therefore, they are only suitable for gene expression data from certain types of cancers / diseases and are unlikely to be applicable to all types of cancers / diseases.

[0039] (4) Methods based on deep neural network models

[0040] Cancer classification methods based on deep neural network models are mainly inspired by the good results achieved by deep models in other fields. Researchers apply autoencoders to feature learning of cancer / disease datasets and then use the learned features in classifiers for classification.

[0041] However, classification methods based on deep models can achieve good results on a small number of gene expression datasets, but this method requires a lot of feature dimensionality reduction time and model training time, and cannot identify characteristic genes with potential biological significance.

[0042] In order to solve the problems arising in the above-mentioned related technologies, the embodiments of the present application provide a classification model construction method. The executor of the classification model construction method can be the classification model construction device provided in the embodiments of the present application, or an electronic device that integrates the classification model construction device. The classification model construction device here can adopt hardware or software. The electronic device can be a smart phone, a tablet computer, a personal calculator, a server or an industrial calculator, etc., which is not limited in the embodiments of the present application.

[0043] Figure 1 Schematic diagram of the process of building a classification model provided in the embodiment of this application Figure 1 ,like Figure 1 As shown, the classification model construction method includes the following steps:

[0044] Step 110: Acquire a target object sample set and extract target feature information of the target object sample set.

[0045] In the embodiments provided herein, the target object may be an object with a small sample size and / or a high sample dimension. Based on the small sample size and high dimensionality of the target object sample set, the classification model establishment method provided in the embodiments of the present application can extract features from the acquired target object sample set to obtain target feature information, thereby eliminating errors and noise in the original target object sample set and removing data irrelevant to the classification results, thereby reducing the dimensionality of the original target object sample set.

[0046] In some embodiments of the present application, the target object may be gene expression data. Specifically, gene expression data refers to the abundance of gene transcription products (mRNA) in cells, measured directly or indirectly in specific tissues using gene chip technology. Gene expression data can be used to analyze which genes have changed in expression, the correlations between genes, and how gene activity is affected under different conditions.

[0047] In the embodiments provided in this application, a gene expression data sample set is composed of multiple gene expression data samples. In practical applications, a gene expression data sample set is expressed by a matrix of m rows and n columns. Referring to Table 1, Table 1 shows an example of a gene expression data sample set; wherein each row in Table 1 represents a sample; the samples in each row can represent a tissue, or a type of cancer / disease, or a subtype of a cancer / disease; (for example, case 1 can be normal human tissue, case 2 can be gastric cancer, case m can be lung cancer, etc.), and each column represents the expression value of the same gene in different cases. Usually n>>m.

[0048] Gene 1 Gene 2 …… Gene n Case 1 219 102.5 …… 45 Case 2 180 117 …… 89 …… …… …… …… …… Situation m 45 56 …… 81

[0049] In the embodiments provided herein, when classifying gene expression data or building a classification model based on a sample set of gene expression data, the original gene expression data contains errors and noise, which can lead to poor classification results. Furthermore, in a certain category of gene expression data, the expression values ​​of most genes have no relationship with the occurrence of cancer / disease, so it is necessary to filter out gene expression data that is closely related to cancer / disease from the gene expression data through some strategies. In addition, the sample size of gene expression data is small and the sample dimension is high. Generally, the sample size is between tens and one hundred, while the sample dimension is between thousands and tens of thousands.

[0050] Based on the above reasons, the classification model establishment method provided in the embodiment of the present application can obtain target feature information by performing feature extraction on the acquired gene expression data sample set; thereby eliminating errors and noise existing in the original gene expression data, removing data that is not related to cancer / disease, and achieving the effect of reducing the dimensionality of the original gene expression data sample set.

[0051] Step 120: Determine multiple split attributes of the target feature information; wherein the split attributes are used to characterize the attributes of the category split nodes in the target object sample set.

[0052] The following description will be made by taking the target object as gene expression data as an example.

[0053] With reference to the gene expression data sample set shown in Table 1, each row is a sample, and each column can be understood as an attribute field for classification. The problem that the present application needs to solve is to establish a cancer / disease classification model based on the gene expression data sample set. The cancer / disease classification model can be a decision tree or a random forest composed of multiple decision trees. When researchers obtain the gene expression data of a certain tissue through gene sequencing technology at some point in the future, they can predict whether the tissue has a lesion and the type of lesion based on the rules corresponding to the random forest composed of the decision tree or multiple decision trees and the expression value attributes of each gene in the tissue.

[0054] In practical applications, the classification model building device can construct one or more trees similar to Figure 2 The decision tree shown, Figure 2 A structural diagram of a decision tree is shown in Figure 2. Figure 2 As shown in the figure, the decision tree has eight nodes: node 1 is the root node, nodes 4, 5, 6, 7, and 8 are leaf nodes, and nodes 2 and 3 are intermediate nodes. Leaf nodes (i.e., nodes 4, 5, 6, 7, and 8) cannot be split by attributes. Intermediate nodes (i.e., nodes 2 and 3) and the root node (i.e., node 1) can be split by attributes. When a node can be split by an attribute, that attribute is called a splitting attribute.

[0055] In the embodiments of the present application, the splitting attribute can be determined by various methods. For example, the classification model building device can determine multiple splitting data of the target feature information by information gain, or by statistical methods. The embodiments of the present application do not limit the method for determining the splitting attribute.

[0056] Step 130: Determine the weight values ​​corresponding to the multiple split attributes, and obtain some target split attributes with the largest weight values ​​from the multiple split attributes.

[0057] The weight value is used to represent the class discrimination of the split attribute. The discrimination of the split attribute specifically refers to the degree to which the split attribute can distinguish the categories of gene expression data.

[0058] In the embodiments provided herein, the classification model construction device can quantify the discrimination and characterize the discrimination of each split attribute by a weight value. A larger weight value indicates a higher discrimination of the split attribute; conversely, a smaller weight value indicates a lower discrimination of the split attribute.

[0059] Exemplary, reference Figure 2 In the decision tree model shown, the attributes of nodes 2 and 3 are both split attributes. However, splitting the attribute of node 2 results in two nodes, while splitting the attribute of node 3 results in three nodes. This shows that the split attribute of node 3 has better discrimination than that of node 2, and the weight of node 3 is greater than that of node 2.

[0060] Furthermore, the classification model construction device can determine the weight value of each split attribute by a variety of methods. In the embodiment provided in the present application, the weight value of each split attribute can be obtained by calculating the Gini coefficient of each split attribute, or by calculating the information gain of each split attribute to obtain the weight value of each split attribute, or by calculating the information gain rate of each split attribute to obtain the weight value corresponding to each split attribute. The embodiment of the present application does not limit the method for determining the weight value of the split attribute.

[0061] In the embodiment provided in the present application, after the weight value of each split attribute is determined, the weight value of each split attribute can be sorted from large to small, and some split attributes with the highest weight values ​​are selected as target split attributes.

[0062] Here, the top 10 to 100 split attributes can be selected as target split attributes, or the top 5% to 10% split attributes can be selected as target split attributes. The embodiment of the present application does not limit the method of selecting the top-ranked split attributes.

[0063] It should be noted that the number of selected split attributes is related to the target object sample set. It can be understood that the number of selected gene split attributes can be adjusted accordingly according to the different target object sample sets. Different target object sample sets will result in different numbers of selected split attributes.

[0064] Step 140: Construct a classification model of the target object based on the target feature information and the target splitting attributes.

[0065] In the embodiment provided in the present application, the classification model construction device can construct a classification model for the target object based on some target splitting attributes selected from multiple splitting attributes and target feature information extracted from the target object sample set. Specifically, the target feature information and the target splitting attributes can be used as inputs of the classification model to train the classification model to obtain a final trained classification model. The classification model here can be a classification model based on a decision tree, a classification model based on a random forest, or a classification model based on a deep forest. The embodiments of the present application are not limited here.

[0066] It can be understood that the classification model construction method provided in the embodiment of the present application does not need to construct a classification model based on all split attributes and the original target object sample set, but rather performs feature extraction on the gene expression data sample set to obtain target feature information; at the same time, selects some split attributes with the largest weight value (i.e., the highest discrimination) from multiple split attributes; and constructs a classification model based on the extracted target feature information and the selected part of the split attributes, which can greatly reduce the data dimension in the classification model training process and shorten the learning and training time of the classification model.

[0067] It can be seen that the classification model construction method provided by the embodiment of the present application obtains a target object sample set, extracts the target feature information of the target object sample set, determines multiple split attributes of the target feature information; determines the weight values ​​corresponding to the multiple split attributes, and obtains some target split attributes with the largest weight value from the multiple split attributes; then, based on the target split attributes and the target feature information, constructs a classification model of the target object. In this way, the target object sample set can be feature extracted to reduce the dimension of the original target object sample set, and at the same time, some split attributes with the largest weight value (i.e., the highest discrimination) are selected from multiple split attributes for model construction, which can greatly reduce the data dimension in the classification model training process and shorten the learning time of the classification model. In this way, by extracting the target feature information after the features are extracted and the selected part of the split attributes for construction of the classification model, the data dimension in the classification model training process can be greatly reduced, thereby improving the training efficiency of the classification model, shortening the training time, and reducing the computational overhead.

[0068] Based on the above embodiment, in the classification model construction method provided in the embodiment of the present application, step 140 can be implemented by steps 1401 and 1402. Specifically, refer to Figure 3 The process of building a classification model is shown in the figure Figure 2 , step 140 specifically includes:

[0069] Step 1401: Use the target feature information and the target split attribute as input to the deep forest model, train and test each layer of the cascade forest in the deep forest model, and obtain a trained deep forest model.

[0070] Step 1402: Use the trained deep forest model as the classification model for the target object.

[0071] In one possible implementation, the classification model mentioned in the embodiments of the present application is a deep forest model. The deep forest model is an integrated forest model that integrates the breadth and depth of the traditional random forest model.

[0072] In practical applications, ensemble learning is a major research direction in the field of machine learning. Its main idea is to combine multiple basic learner models into a strong classifier to improve the classification effect. According to relevant research, in order to improve the accuracy and generalization of the ensemble learning model, we can start from two aspects. First, we must ensure the diversity and representativeness of the features; second, we must ensure the diversity of the basic learners. In recent years, a new ensemble learning model has been proposed. This model draws on the idea of ​​multi-layer feature extraction in deep learning, and expands the random forest into a multi-layer connected forest model to improve the classification effect. The multi-layer connected forest model is also called a deep forest model. In the embodiments provided in this application, the deep forest model is used in the classification problem of target objects with a small number of samples and a high sample dimension, such as the classification problem of cancer / related target objects, to improve the accuracy of cancer / disease classification.

[0073] In the embodiments provided herein, a deep forest model may include N layers of cascade forests, where N is an integer greater than 1. Each layer of the cascade forest in the deep forest model includes M random forests, and each random forest includes L decision trees.

[0074] Specifically, the classification model construction device can pre-configure the initial values ​​of the deep forest model's hyperparameters. Hyperparameters refer to parameters whose values ​​are set before the classification model is trained, rather than parameter data obtained through training. In the embodiments provided herein, the hyperparameters of the deep forest model include at least the maximum number of layers N of the cascade forest, the number M of random forests in each layer of the cascade forest, and the number L of decision trees in each random forest.

[0075] In one possible implementation, before using the target feature information and the target split attribute as input to the deep forest model, the following steps are further included:

[0076] Step 1400a: Receive configuration information for the deep forest model;

[0077] Step 1400b: Determine the maximum number of cascade forest layers N included in the deep forest model, the number of random forests M in each cascade forest layer, and the number of decision trees L in each random forest based on the configuration information.

[0078] It is understandable that the classification model building device can receive the configuration information for the deep forest model input by the user, or the configuration information of the deep forest model sent by other third-party platforms, and configure the maximum number of layers of cascade forests included in the deep forest model, the number of random forests in each layer of cascade forests, and the number of decision trees in each random forest according to the configuration information.

[0079] Furthermore, after configuring the initial values ​​of the deep forest model hyperparameters, the target feature information obtained in step 110 and the target splitting attributes determined in step 130 are used as inputs of the deep forest model to train and test the deep forest model to obtain a trained deep forest model.

[0080] Based on the above embodiment, in the classification model construction method provided in the embodiment of the present application, step 1401 can be implemented through steps 1401a to 1401c. Figure 4 The classification model construction method shown is as follows Figure 3 Step 1401 specifically includes the following steps:

[0081] Step 1401a: Input the target feature information into the first-level cascade forest of the deep forest model, train and test the first-level cascade forest, and obtain the first category vector;

[0082] Step 1401b: Using the i-th category vector and the split feature vector corresponding to the target split attribute as input to the i+1-th cascade forest, training and testing the i+1-th cascade forest to obtain the i+1-th category vector; wherein the split feature vector is used to represent the feature vector of the target feature information that is classified into the target split attribute;

[0083] Step 1401c: Continue using the i+1th category vector and the split feature vector corresponding to the target split attribute as the input of the i+2th cascade forest, and train and test the i+2th cascade forest until the training and testing of the Nth cascade forest are completed; where i is an integer greater than or equal to 1 and less than N-1.

[0084] In practical applications, due to the characteristics of small sample data volume and high sample dimension of the target object, the classification effect is poor under simple classification models (such as KNN classification model, classification model based on autoencoder, etc.); and when the sample dimension is high, it is very easy to fall into overfitting.

[0085] Based on this, the embodiment of the present application comprehensively considers the relationship between the complexity of the classification model and the amount of data and the data dimension, so that the data dimension matches the model complexity. Generally speaking, the deep forest model will adopt a multi-granularity scanning strategy for the original sample set. This approach may be effective for samples with sufficient sample data and low data dimension; however, the use of a multi-granularity scanning strategy is "adding insult to injury" for target objects with small sample data amounts and high sample dimensions; it will aggravate the mismatch between data dimension and model complexity. Therefore, the classification model construction method provided in the embodiment of the present application, in order to make the deep forest model more suitable for the sample characteristics of the target object, only retains the cascade forest characteristics of the deep forest model, and abandons the multi-granularity scanning characteristics of the deep forest model.

[0086] In addition, in order to further improve the accuracy, the embodiment of the present application makes modifications to the transfer vectors between cascade forests in the deep forest model. Figure 5 The structural diagram of the deep forest model is shown in the figure, which describes the training process of the deep forest in detail. Figure 5 It is exemplarily shown that each layer of the cascade forest of the deep forest model includes four random forests, namely random forest 1, random forest 2, random forest 3 and random forest 4.

[0087] Specifically, refer to Figure 5 The classification model construction device inputs the target feature vector into the first-level cascade forest 501 of the cascade random forest. In this way, each random forest in the first-level cascade forest 501 calculates the percentage of different categories at the node where the target feature information falls, and then calculates the average value of all trees in each random forest to generate a category estimate. The estimated categories respectively form the first category vector 502.

[0088] Next, the first category vector 502 output by the first-level cascade forest 501 is combined with the split feature vector 504 corresponding to the target split attribute 503 selected from multiple split attributes, and used as the input to the second-level cascade forest 505. Furthermore, each random forest in the second-level cascade forest 505 calculates the percentage of different categories at the node where the input data falls, and then calculates the average value of all trees in each random forest to obtain the second category vector 506.

[0089] It is worth noting that the split feature vector 504 corresponding to the target split attribute 503 is the set of all features in the target feature information that are classified under the target split attribute. Here, the first category vector 502 output by the first-level cascade forest 501 is combined with the split feature vector 504 corresponding to the target split attribute 503. Specifically, the first category vector 502 output by the first-level cascade forest 501 and the split feature vector 504 corresponding to the target split attribute 503 are concatenated.

[0090] Furthermore, the vector formed by combining the second category vector 506 and the split feature vector 504 corresponding to the target split attribute 503 is used as the input to the third layer of the cascade forest (not shown in the figure) to obtain the third category vector. The vector formed by combining the third category vector and the split feature vector 504 corresponding to the target split attribute 503 is used to train and test the next layer of the cascade forest until the training and testing of each layer of the cascade forest in the deep forest model is completed; the Nth category vector obtained in the Nth layer is averaged and the maximum value is taken to obtain the final trained deep forest model, which is used to predict the category of the target object.

[0091] This can be understood as the classification model building device sorting the split attributes of the target object sample set by weight, selecting the largest split attributes and combining them with the category vector output by each cascade forest layer to form a new vector, which is then passed to the next cascade forest layer. In this way, selecting only the largest split attributes for classification model training can greatly reduce the training data dimension and, to a certain extent, shorten training time. Moreover, compared to the original target object set, where a single sample contains thousands or tens of thousands of characteristic genes, selecting split feature information corresponding to the more discriminatory split attributes can better assist in clinical judgment of which genes are associated with a particular cancer and help discover biologically significant characteristic genes.

[0092] Based on the above embodiment, in the classification model construction method provided in the embodiment of the present application, after step 110 obtains the target object sample set, the following steps are further included:

[0093] Divide the target object sample set into K mutually disjoint subsets;

[0094] Select K-1 of the K subsets as training sample sets, and select the remaining subsets as test sample sets;

[0095] Correspondingly, step 1401 uses the target feature information and the target split attribute as input to the deep forest model, trains and tests each random forest layer in the deep forest model, and obtains a trained deep forest model, including the following steps:

[0096] Each layer of the cascade forest of the deep forest model is trained using the target feature information and target splitting attributes corresponding to the training sample set to obtain the initial deep forest model;

[0097] By testing the target feature information and target splitting properties corresponding to the sample set, each layer of the cascade forest of the initial deep forest model is tested and adjusted to obtain a trained deep forest model.

[0098] In practical applications, researchers generally divide the original dataset into a training set and a test set. The training set is used to train the model, and the test set is used to evaluate the model's generalization ability, that is, its adaptability to new samples. In the examples of this application, based on the small amount of data in the target object sample set, the K-fold cross-test principle is used to train and test the deep forest model.

[0099] Specifically, the target object sample set is divided into K mutually non-overlapping subsets; K-1 subsets are selected from the K subsets as training sample sets, and the remaining subsets are selected as test sample sets.

[0100] Next, each layer of the cascade forest of the deep forest model is trained using the target feature information and target splitting attributes corresponding to the training sample set to obtain an initial deep forest model. Further, each layer of the cascade forest of the initial deep forest model is tested and updated using the target feature information and target splitting attributes corresponding to the test sample set to obtain a trained deep forest model.

[0101] It should be noted that the training and testing of each layer of the cascade forest in the deep forest model are the same as in step 1041a, step 1041b, and step 1041c, and will not be repeated here.

[0102] Based on the above embodiment, the target object in the embodiment of the present application can be gene expression data. Based on this, the classification model construction method provided in the embodiment of the present application, before step 110 of obtaining the target object sample set, further includes the following steps:

[0103] Step 101: Acquire multiple gene expression data;

[0104] Step 102: pre-process the plurality of gene expression data to obtain a target object sample set; wherein the pre-processing includes: performing missing value processing and / or standardization processing on the gene expression data.

[0105] In the examples provided herein, the gene expression data obtained from the gene chip contain missing values, so the missing values ​​need to be processed. At the same time, in order to improve the overfitting of the trained classification model, the gene expression data generally needs to be normalized.

[0106] Based on this, after obtaining multiple gene expression data, the classification model construction device needs to preprocess each gene expression data; the preprocessing mainly includes: missing value processing and / or standardization processing. Among them, the processing of missing values ​​is specifically: if the number of missing values ​​is greater than the preset threshold, the gene expression data sample is discarded; if the number of missing values ​​is less than the preset threshold, the missing value is filled. Here, there are many ways to fill the missing values. In a feasible embodiment, the multiple gene expression data can be arranged in the format described in Table 1, the mean of the column where the missing value is located is calculated, and the obtained mean is filled into the missing position.

[0107] In addition, the gene expression data is normalized, specifically mapping the values ​​in the gene expression data to between (0, 1), also known as normalization. Finally, the pre-processed gene expression data are arranged into the format described in Table 1 to obtain the target object sample set.

[0108] In a feasible implementation, step 110 extracts target feature information of the target object sample set, specifically including:

[0109] Target feature information of the target object sample set is extracted according to a preset feature selection method; wherein the preset feature selection method is used to reduce the dimension of the target object.

[0110] Since the dimension of the target object sample set is relatively high, the embodiment of the present application performs dimensionality reduction by extracting target feature information of the target object sample set.

[0111] Specifically, the preset feature selection method includes at least one of the following:

[0112] T-test method, Fisher discriminant method, category-related feature method, and genetic algorithm.

[0113] The classification model construction method provided in the embodiment of the present application is to sort the split attributes of the target object sample set by weight, select the largest part of the split attributes and combine them with the category vector output by each layer of cascade forest to form a new vector, thereby passing it to the cascade forest of the next layer. In this way, only selecting the largest part of the split attributes for training the classification model can greatly reduce the data dimension of the training and alleviate the training time to a certain extent; and, compared with the original target object set in which a sample contains thousands or tens of thousands of features, the split feature information corresponding to the split attribute with higher discrimination is selected to improve the accuracy of classification. It is understandable that when the target object is gene expression data, the classification model construction method provided by the present application is more helpful for clinical judgment of which genes are related to a certain disease / cancer, and is helpful for discovering characteristic genes with biological significance.

[0114] Below, the classification model construction method provided in the embodiment of the present application is described in detail in combination with specific scenarios.

[0115] Based on the above embodiments, the classification model construction method provided in the embodiments of the present application can be applied to Figure 6 The system architecture diagram shown in Figure 1 is shown in Figure 1. Figure 6 As shown, the classification model construction method provided in the embodiment of the present application specifically includes two stages: the model training stage and the model testing stage. Each stage includes five steps: gene expression data preprocessing, K-fold cross-test partitioning, target feature information selection, classification model training (classification model testing), and classification model evaluation. The specific content of each step is described in detail below.

[0116] (1) Gene expression data preprocessing.

[0117] Gene expression data acquired from gene chips contain missing values, so these need to be processed. Furthermore, to mitigate overfitting in trained models, data standardization is generally required. Therefore, preprocessing of gene expression data involves both missing value processing and standardization. For missing values, the mean of the feature column containing the missing value is used to fill the gap. If there are too many missing values, the sample is discarded. Data standardization involves mapping the gene expression data values ​​to a range between (0, 1).

[0118] (2) K-fold cross-test partitioning. Cross-testing is a method based on statistical thinking to divide data samples into multiple subsets. In general data mining and machine learning research, researchers generally divide the original data set into two parts: a training set and a test set. The training set is used to train the model, and the test set is used to evaluate the generalization ability of the model. However, due to the small sample size of gene expression data, such partitioning often leads to inaccurate results. In the embodiment of the present application, the K-fold cross-test principle is used to train the deep forest model and test the deep forest model.

[0119] Specifically, the gene expression data sample set is divided into K mutually disjoint subsets;

[0120] Select K-1 of the K subsets as training sample sets, and select the remaining subsets as test sample sets.

[0121] (3) Target feature information selection.

[0122] Since the dimensionality of gene expression data samples is relatively high, the embodiment of the present application performs dimensionality reduction by extracting features from the gene expression data sample set to obtain target feature information.

[0123] (4) Classification model training and classification model testing.

[0124] In the embodiment provided in the present application, for the classification model training process, the classification model construction device can first determine multiple split attributes of the target feature information corresponding to the training sample set; then determine the weight value corresponding to each split attribute in the test sample set; and obtain the target split attributes with the largest weight value from the multiple split attributes; input the target feature information and target split attributes corresponding to the training sample set into the deep forest model for training, thereby obtaining an initial deep forest model;

[0125] Furthermore, after obtaining the initial deep forest model, the initial deep forest model is trained. Here, the classification model construction device can determine multiple split attributes of the target feature information corresponding to the test sample set; then determine the weight value corresponding to each split attribute in the test sample set, and obtain the target split attributes with the largest weight value from the multiple split attributes; input the target feature information and target split attributes in the test sample set into the initial deep forest model, update and adjust the parameters of the initial deep forest, obtain a trained deep forest model, and use the trained deep forest model as the classification model for the gene expression data.

[0126] It should be noted that for the training process of the training sample set and the testing process of the test sample set, it is necessary to first determine the splitting attributes corresponding to the training sample set or the test sample set, select the first X attributes and the category vector output by each level of the deep forest model or the initial deep forest model to combine into a new vector, and then pass it to the cascade forest of the next layer; thereby completing the training and testing of each layer of the cascade forest in the deep forest model or the initial deep forest model. This can alleviate the training time to a certain extent and help discover characteristic genes with biological significance. In the embodiments provided in this application, X can take a value between 10 and 100. The specific value of X can be adjusted accordingly according to the different gene expression data sample sets. Different gene expression data sample sets have different corresponding number of splitting attributes.

[0127] It's important to note that some initial hyperparameter values ​​must be set before training. For example, the number of random forests in each cascade forest layer, the number of decision trees in each random forest, and the number of cascaded layers in the Deep Forest model. Since the improved Deep Forest model lacks multi-granularity scanning, these three hyperparameters are crucial.

[0128] (5) Classification model evaluation. Classification model evaluation is used to evaluate the accuracy and reliability of the classification model. Common evaluation indicators include classification prediction accuracy, area under the receiver operating characteristic (ROC) curve (AUC) value, confusion matrix, etc. However, in cancer / disease classification problems, the most commonly used are classification accuracy and recall.

[0129] It can be seen from this that the classification model construction method provided by the embodiment of the present application, by obtaining a gene expression data sample set, and extracting the target feature information of the gene expression data sample set, then determining multiple split attributes of the target feature information; determining the weight values ​​corresponding to multiple split attributes respectively, and obtaining the partial target split attributes with the largest weight value from multiple split attributes; the weight value is used to characterize the category discrimination of the split attribute; based on the target split attribute and the target feature information, a classification model of gene expression data is constructed. In this way, feature extraction can be performed on the gene expression data sample set, reducing the dimension of the original gene expression data sample set, and at the same time, selecting the partial split attributes with the largest weight value (i.e., the highest discrimination) from multiple split attributes for model construction can greatly reduce the data dimension in the classification model training process and shorten the learning time of the classification model. In this way, the target feature information after feature extraction and the selected partial split attributes are used to construct the classification model, which can greatly reduce the data dimension in the classification model training process, thereby improving the training efficiency of the classification model, shortening the training time, and reducing computational overhead.

[0130] Based on the above embodiments, the present application provides a classification model construction device, such as Figure 7 As shown, the device includes:

[0131] The feature extraction unit 71 is used to obtain a target object sample set and extract target feature information of the target object sample set;

[0132] A split attribute determination unit 72 is configured to determine a plurality of split attributes of the target feature information; wherein the split attributes are used to characterize the attributes of the category split nodes in the target object sample set;

[0133] The weight value determining unit 73 is configured to determine weight values ​​corresponding to the plurality of split attributes, and obtain a portion of target split attributes with the largest weight value from the plurality of split attributes; the weight value is used to represent the class discrimination of the split attribute;

[0134] The processing unit 74 is configured to establish a classification model of the target object based on the target splitting attribute and the target feature information.

[0135] Optionally, the classification model is a deep forest model, wherein the deep forest model includes N layers of cascade forests; N is an integer greater than 1;

[0136] The processing unit 74 is specifically used to use the target feature information and the target splitting attribute as input to the deep forest model, and train and test each layer of the cascade forest in the deep forest model to obtain a trained deep forest model; and use the trained deep forest model as the classification model of the target object.

[0137] Optionally, the processing unit 74 is further used to input the target feature information into the first layer cascade forest of the deep forest model, train and test the first layer random forest to obtain the first category vector; use the i-th category vector and the split feature vector corresponding to the target split attribute as the input of the i+1-th layer cascade forest, train and test the i+1-th layer cascade forest to obtain the i+1-th category vector; wherein the split feature vector is used to represent the feature vector divided into the target split attribute in the target feature information; continue to use the i+1-th category vector and the split feature vector corresponding to the target split attribute as the input of the i+2-th layer cascade forest, train and test the i+2-th layer random forest, until the training and testing of the N-th layer cascade forest is completed; wherein i is an integer greater than or equal to 1 and less than N-1.

[0138] Optionally, the classification model building device further includes an acquisition unit 75 for receiving configuration information for the deep forest model;

[0139] The processing unit 74 is configured to determine, based on the configuration information, a maximum number N of cascade forest layers included in the deep forest model, a number M of random forests in each cascade forest layer, and a number L of decision trees in each random forest.

[0140] Optionally, the target object includes gene expression data.

[0141] Optionally, the acquisition unit 75 is further configured to acquire a plurality of gene expression data;

[0142] The processing unit 74 is further configured to pre-process the plurality of gene expression data to obtain the target object sample set; the pre-processing includes: performing missing value processing and / or standardization processing on the gene expression data.

[0143] Optionally, the feature extraction unit 71 is configured to extract target feature information of the target object sample set according to a preset feature selection method; wherein the preset feature selection method is used to reduce the dimension of the target object.

[0144] Optionally, the preset feature selection method includes at least one of the following:

[0145] T-test method, Fisher discriminant method, category-related feature method, and genetic algorithm.

[0146] Optionally, the processing unit 74 is used to divide the target object sample set into K mutually non-overlapping subsets; wherein K is an integer greater than 1; select K-1 from the K subsets as training sample sets, and select the remaining subsets as test sample sets; train each layer of the cascade forest of the deep forest model through the target feature information and target splitting attributes corresponding to the training sample set to obtain an initial deep forest model; test and update each layer of the cascade forest of the initial deep forest model through the target feature information and target splitting attributes corresponding to the test sample set to obtain a trained deep forest model.

[0147] It can be seen from this that the classification model construction device provided by the embodiment of the present application, by obtaining the target object sample set, and extracting the target feature information of the target object sample set, determines multiple split attributes of the target feature information; determines the weight values ​​corresponding to multiple split attributes respectively, and obtains the partial target split attributes with the largest weight value from multiple split attributes; the weight value is used to characterize the category discrimination of the split attribute; based on the target split attribute and the target feature information, constructs a classification model of the target object. In this way, the classification model construction device can extract features from the gene expression data sample set, reduce the dimension of the original target object sample set, and at the same time select the partial split attributes with the largest weight value (i.e., the highest discrimination) from multiple split attributes for model construction, which can greatly reduce the data dimension in the classification model training process and shorten the learning time of the classification model. In this way, by extracting the target feature information after the feature and the selected partial split attributes for construction of the classification model, the data dimension in the classification model training process can be greatly reduced, thereby improving the training efficiency of the classification model, shortening the training time, and reducing the computational overhead.

[0148] Based on the implementation of each unit in the above-mentioned image classification device, in order to implement the classification model construction method provided in the embodiment of the present application, the embodiment of the present application also provides an electronic device, such as Figure 8 As shown, the electronic device 80 includes: a processor 81 and a memory 82 configured to store a computer program that can be run on the processor.

[0149] The processor 81 is configured to execute the method steps in the aforementioned embodiment when running the computer program.

[0150] Of course, in actual application, Figure 8 As shown, the various components in the electronic device 80 are coupled together via a bus system 83. It is understood that the bus system 83 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 83 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 8 The various buses are collectively labeled bus system 83 .

[0151] In an exemplary embodiment, the present application also provides a computer-readable storage medium, for example comprising Figure 8 The computer program memory 82 shown is executable by the processor 81 of the electronic device 80 to complete the steps of the aforementioned method. The computer-readable storage medium may be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface mount storage device, an optical disc, or a compact disc read-only memory (CD-ROM).

[0152] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0153] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0154] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0155] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0156] The above description is merely a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application.

Claims

1. A classification model construction method, characterized in that: The method comprises: Obtain a sample set of target objects; Extracting target feature information of the target object sample set according to a preset feature selection method; wherein the preset feature selection method is used to reduce the dimension of the target object; the target object includes gene expression data; Determining a plurality of splitting attributes of the target feature information; wherein the splitting attributes are used to characterize the attributes of the category splitting nodes in the target object sample set; Determine weight values ​​corresponding to the multiple split attributes respectively, and obtain some target split attributes with the largest weight values ​​from the multiple split attributes; the weight values ​​are used to represent the category discrimination of the split attributes; A classification model of the target object is constructed based on the target feature information and the target splitting attribute.

2. The method according to claim 1, characterized in that The classification model is a deep forest model, wherein the deep forest model includes N layers of cascade forests; N is an integer greater than 1; The constructing of a classification model for the target object based on the target splitting attribute and the target feature information includes: Using the target feature information and the target split attribute as input to a deep forest model, and training and testing each layer of the cascade forest in the deep forest model to obtain a trained deep forest model; The trained deep forest model is used as the classification model of the target object.

3. The method according to claim 2, characterized in that The target feature information and the target split attribute are used as inputs of a deep forest model, and each layer of the cascade forest in the deep forest model is trained and tested to obtain a trained deep forest model, including: Inputting the target feature information into the first layer of the cascade forest of the deep forest model, training and testing the first layer of random forest to obtain a first category vector; The i-th category vector and the split feature vector corresponding to the target split attribute are used as inputs of the i+1-th layer cascade forest, and the i+1-th layer cascade forest is trained and tested to obtain the i+1-th category vector; wherein the split feature vector is used to represent the feature vector of the target feature information that is divided into the target split attribute; Continue to use the i+1th category vector and the split feature vector corresponding to the target split attribute as the input of the i+2th layer cascade forest, and train and test the i+2th layer random forest until the training and testing of the Nth layer cascade forest is completed; where i is an integer greater than or equal to 1 and less than N-1.

4. The method according to claim 2 or 3, characterized in that Before using the target feature information and the target split attribute as input to the deep forest model, the method further includes: Receiving configuration information for the deep forest model; Based on the configuration information, the maximum number N of cascade forest layers included in the deep forest model, the number M of random forests in each layer of the cascade forest, and the number L of decision trees in each random forest are determined.

5. The method according to any one of claims 1 to 3, characterized in that Before obtaining the target object sample set, the method includes: Obtain multiple gene expression data; Preprocessing is performed on the plurality of gene expression data to obtain the target object sample set; the preprocessing includes: performing missing value processing and / or standardization processing on the gene expression data.

6. The method according to any one of claims 1 to 3, characterized in that The preset feature selection method includes at least one of the following: T-test method, Fisher discriminant method, category-related feature method, and genetic algorithm.

7. The method according to claim 2, characterized in that After obtaining the target object sample set, the method further includes: Dividing the target object sample set into K mutually disjoint subsets; wherein K is an integer greater than 1; Select K-1 of the K subsets as training sample sets, and select the remaining subsets as test sample sets; The target feature information and the target split attribute are used as inputs of a deep forest model, and each random forest layer in the deep forest model is trained and tested to obtain a trained deep forest model, including: Training each cascade forest layer of the deep forest model using target feature information and target splitting attributes corresponding to the training sample set to obtain an initial deep forest model; Each layer of the cascade forest of the initial deep forest model is tested and updated using the target feature information and target splitting attributes corresponding to the test sample set to obtain a trained deep forest model.

8. A classification model building device, characterized in that: The device comprises: a feature extraction unit, configured to obtain a target object sample set and extract target feature information of the target object sample set according to a preset feature selection method; wherein the preset feature selection method is configured to reduce the dimension of the target object; and the target object includes gene expression data; a split attribute determination unit, configured to determine a plurality of split attributes of the target feature information; wherein the split attributes are used to characterize the attributes of the category split nodes in the target object sample set; a weight value determining unit, configured to determine weight values ​​corresponding to the plurality of split attributes, and to obtain a portion of target split attributes with the largest weight value from the plurality of split attributes; the weight value is used to characterize the class discrimination of the split attribute; A processing unit is used to build a classification model of the target object based on the target feature information and the target splitting attribute.

9. An electronic device comprising a processor and a memory for storing a computer program capable of running on the processor; in, When the processor is used to run the computer program, it executes the steps of the classification model construction method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the classification model construction method according to any one of claims 1 to 7.

Images