Hierarchical classification method for high-dimensional biomedical data based on multi-path feature selection

Abstract

This invention proposes a hierarchical classification method for high-dimensional biomedical data based on multi-path feature selection, primarily addressing the low classification accuracy of existing methods for high-dimensional biomedical data. The implementation scheme involves: acquiring a high-dimensional biomedical dataset, performing preprocessing such as padding, encoding, and standardization, and then partitioning it; forming a tree-like hierarchical structure of the dataset based on medical etiological classification systems; and using L... 1,2 Norm regularization is used for hierarchical feature selection within a tree-like hierarchical structure. An existing logistic regression classifier is selected as the classifier in the hierarchical structure and trained. Test set samples are input into the trained hierarchical classifier for classification prediction and output. This invention reduces the difficulty of modeling, effectively removes redundant features, improves computational efficiency, and alleviates the inherent inter-layer error propagation in existing hierarchical classification methods. It improves the classification accuracy of high-dimensional biomedical data and can be used to provide a reference for disease diagnosis in medicine.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and specifically relates to a high-dimensional biomedical data classification method, which can be used to provide a reference for disease diagnosis in medicine. Background Technology

[0002] With the rapid development of information technologies such as big data, artificial intelligence, and cloud computing, the amount of data across all industries has increased dramatically, ushering in the era of big data. "Data explosion and knowledge shortage" has become a true reflection of the information age, and how to discover valuable information from this ever-increasing volume of data has become a research hotspot in fields such as statistics, machine learning, and artificial intelligence. Medical data, which directly impacts people's lives, is a key focus. With the development of medical technology and the rapid improvement of medical informatization, hospitals have accumulated a vast amount of medical data during treatment, and this data exists in various forms. How to extract valuable information from this big data and how to predict future trends are of significant reference value to relevant medical workers and researchers.

[0003] A key characteristic of biomedical data is its small sample size and numerous features. While the sample size may only be a few dozen, it can contain tens of thousands of features. These features include a large number of irrelevant or weakly correlated ones. The presence of these features not only increases the computation time for classification but also reduces classification accuracy, posing significant challenges to data analysis and potentially leading to the so-called "curse of dimensionality." Furthermore, classification and prediction models built from such data are highly susceptible to overlearning, and the large number of categories in biomedical data makes modeling even more difficult.

[0004] With the rapid development of machine learning technology, the application of machine learning to biomedical data research has attracted researchers' interest. Decision trees, support vector machines, artificial neural networks, and K-nearest neighbor classification are all commonly used methods for processing biomedical data. However, these flat classification methods that directly address all categories experience a decrease in accuracy as the number of categories increases, and they do not perform feature selection to remove redundant features, thus failing to meet application requirements.

[0005] For classification problems involving numerous classes, hierarchical classification has been proposed and successfully applied in data classification research. Hierarchical classification is a task that utilizes the hierarchical relationship between data categories to classify data. It employs a divide-and-conquer strategy, breaking down a large-scale global classification problem into smaller, local classification problems according to the category hierarchy, enabling efficient organization and processing of large-scale data.

[0006] In their paper "Joint hierarchical category structure learning and large-scale image classification" (IEEE Transactions on Image Processing, 2017, 26(9):4331–4346), Qu et al. used an optimal path algorithm to transform the category prediction problem into a multi-path search problem. The algorithm requires traversing the entire hierarchical tree structure, calculating the marginal probabilities of all fine-grained classes within a coarse-grained category, and finding the path with the highest joint probability for prediction. While this method alleviates the error propagation problem between layers to some extent, the classification result is still based on probability, resulting in limited improvement in classification accuracy.

[0007] In their paper "Cost-sensitive hierarchical classification for imbalance classes" (Applied Intelligence, 2020: 1–16), Zheng Weijie et al. proposed a cost-sensitive hierarchical classification method for imbalanced data distribution. This method constructs a cost-sensitive factor to balance the relationship between the majority and minority classes. A threshold is set to judge class probabilities; when the prior probability value of a granular class does not meet the threshold, a node is penalized. This classification method balances the differences between the majority and minority classes, improving the classification accuracy of the minority class, but simultaneously reducing the classification accuracy of the majority class, resulting in a relatively small overall improvement in classification accuracy. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of the existing technology by proposing a hierarchical classification method for high-dimensional biomedical data based on multi-path feature selection, so as to improve the accuracy of classification and enhance the overall classification precision.

[0009] The technical solution to achieve the objective of this invention is as follows: by extracting effective features from high-dimensional medical data, organizing the data into a hierarchical structure according to disease characteristics for hierarchical classification, and performing prediction from top to bottom until the final classification category is obtained. The implementation steps include the following:

[0010] (1) Obtain publicly available high-dimensional biomedical data or medical datasets compiled with data from cooperating hospitals, perform preprocessing such as missing value imputation, encoding and standardization in sequence to form standard data, and divide it into training set and test set in a 9:1 ratio;

[0011] (2) Form a tree-like hierarchical structure for the dataset based on the medical etiology classification system;

[0012] (3) Based on L 1,2 Norm regularization performs hierarchical feature selection at each level of the tree-like hierarchical structure:

[0013] (3a) Let ω be the weight matrix of each granularity layer. i For its two components W i With D i sum:

[0014]

[0015]

[0016] ω i j =w i j +d i j

[0017]

[0018] Where i = 1, 2, ..., l, l is the level of the tree structure, and j = 1, 2, ..., m i m i Let ω be the number of categories in the i-th granular layer, n be the dimension of the sample features, and ω be the number of categories in the granular layer. i j The weight matrix ω i The j-th category vector, w i j The weight matrix ω i Disassembled W i The j-th class vector of the matrix, d i j The weight matrix ω i Disassembled W i The j-th category vector of the matrix;

[0019] (3b) ω i 、W i and D i Substituting these three parameters into the feature selection algorithm of the sparse representation model, we obtain information about W. i D i The loss function J(W) that needs to be optimized is minimized. i D i ):

[0020]

[0021] in Let be the sample matrix of the i-th layer belonging to the j-th class. The corresponding weight vector is ω ij , Sample matrix The number of samples, Sample matrix The corresponding category label, |||| 1,2 It refers to L 1,2 Norms, λ1, and λ2 are nonnegative parameters that control regularization;

[0022] (3c) Use accelerated gradient descent to minimize the loss function J(W) i D i The solution is iteratively solved until J(W) is reached. i D i The convergence is obtained in k iterations. The optimal solution W i (*) ,

[0023] (3d) The optimal solution W generated by k iterations i (*) , By adding them together, we obtain the optimal weight matrix ω. i (*) and for ω i (*) The element values ​​are sorted, and the first N features are selected as the N features selected for each layer of the tree structure according to the sorting order.

[0024] (4) Select an existing logistic regression classifier as the classifier in the hierarchical structure and train it.

[0025] (4a) Based on its tree-like hierarchical structure, select N features selected in step (3d) at each level of the training set, and use these features to form a feature subset of the training set;

[0026] (4b) Iteratively update the two parameters in the logistic regression classifier using a subset of the training set via gradient descent. and deviation Until its loss function converges, a well-trained logistic regression classifier model is obtained, where... The feature weights of the sample data belonging to the j-th class of the i-th layer are... This represents the deviation of the j-th class in the i-th layer;

[0027] (5) Perform classification prediction on the medical data samples to be classified in the test set and output the results:

[0028] (5a) Input the test set samples into the trained hierarchical classifier to obtain the probability that the test sample belongs to the j-th class of the i-th layer of the hierarchical structure.

[0029]

[0030] Where T represents the pair Transpose of; These are the feature weights of the test sample data belonging to the j-th class in the i-th layer. This represents the bias of the test sample data belonging to the j-th class of the i-th layer. These two values ​​are different from those obtained from training the training samples. and equal;

[0031] (5b) Select the first k pre-selected classes in each layer according to the class probability value, until the first k pre-selected classes in the leaf node layer of the lowest layer of the tree hierarchy are obtained;

[0032] (5c) Use an SVM classifier in the k pre-selected classes of the leaf layer to output the final class of the test sample.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] First, for high-dimensional biomedical multi-class problems, this invention adopts a hierarchical classification method, which decomposes the multi-class problem into multiple sub-class learning tasks, effectively reducing the difficulty of modeling, and the computational overhead of classification learning and prediction is relatively small, which can effectively meet the classification needs of biomedical data.

[0035] Second, based on hierarchical classification, this invention constructs a multi-path prediction model. First, it uses a top-down recursive method to calculate the probability of hierarchical classes through logistic regression classification. Second, it calculates the probability of the current class by combining the probabilities of the parent class and the subclasses to which the current parent class belongs, and selects multiple possible fine-grained classes at the current level based on their sibling relationships. Finally, it uses an SVM classifier for final judgment. Compared with existing methods, this alleviates the inherent problem of inter-layer error propagation in hierarchical classification and reduces the possibility of misclassification from higher layers.

[0036] Third: In the process of hierarchical classification, this invention adopts l 1,2 Norm regularization constraints select the optimal feature subset to remove redundancy and interference, since L 1,2 It is an unbiased estimate, and therefore, compared with the L1 minimization problem, the solution obtained by this invention has stronger sparsity and higher computational efficiency. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the implementation of the present invention.

[0038] Figure 2 This is a schematic diagram of the tree-like hierarchical structure for classifying medical datasets in this invention. Detailed Implementation

[0039] The embodiments and effects of the present invention will be described in further detail below with reference to the accompanying drawings.

[0040] Reference Figure 1 The implementation steps of this example include the following:

[0041] Step 1: Acquire biomedical data and preprocess and segment it.

[0042] 1.1) Obtain publicly available high-dimensional biomedical datasets from the UCI database or obtain high-dimensional biomedical datasets compiled from data of partner hospitals;

[0043] 1.2) Preprocess the biomedical dataset to form a standard dataset:

[0044] 1.2.1) Missing value imputation, that is, using KNN interpolation to imput continuous missing data, and using the mode of all data of the data item to imput discrete missing data;

[0045] 1.2.2) Encoding, which involves selecting class features that have no magnitude significance for the discrete data after missing value imputation, and using numerical or vectorized operations to convert them into 0-1 codes and one-hot codes that computers can recognize:

[0046] This 0-1 encoding uses binary variables 0 and 1 to represent the features of each sample across two categories;

[0047] This one-hot encoding adds each category with more than three categories as a new feature to the dataset;

[0048] 1.2.3) Standardization, which involves normalizing the continuous data after imputation of missing values ​​to distribute it in the interval [0,1], resulting in preprocessed standardized data x′:

[0049]

[0050] Where x represents the original continuous data, MIN represents the minimum value of the data item containing x, and MAX represents the maximum value of the data item containing x;

[0051] 1.3) Divide the standardized dataset into training and test sets in a 9:1 ratio.

[0052] Step 2: Obtain the tree-like hierarchical structure of the training and test sets.

[0053] Based on the etiological classification system corresponding to the medical dataset, a tree-like hierarchical structure is formed for the training and test sets, such as... Figure 2 As shown;

[0054] The tree-like hierarchical structure is a triangular tree structure comprising multiple levels of category clusters, wherein:

[0055] There is only one root node at the top level, which is the parent node of all nodes.

[0056] The intermediate layer contains multiple category cluster layers, the number of which is determined according to the characteristics of the corresponding medical disease in each medical dataset. Each category cluster layer contains multiple nodes.

[0057] The bottom layer has multiple nodes, which are called leaf nodes;

[0058] Each node in each level represents a category, and the leaf nodes are the final classification categories that need to be determined. The entire tree hierarchy runs from the root node to the leaf nodes from top to bottom, corresponding to different etiological categories.

[0059] In this example, the tree structure has a hierarchical structure with one, but not limited to, intermediate layers. The top layer has a root node, the intermediate layers have three nodes, and the bottom layer has six nodes. Each node corresponds to a different etiology category, and the final classification result is the six etiology categories at the bottom layer.

[0060] Step 3, based on L 1,2 Norm regularization performs hierarchical feature selection at each level of the tree-like hierarchical structure.

[0061] 3.1) Let ω be the weight matrix for each granularity layer. i For its two components W i With D i sum:

[0062]

[0063]

[0064] ω i j =w i j +d i j

[0065]

[0066] Where i = 1, 2, ..., l, l is the level of the tree structure, and j = 1, 2, ..., m i m i Let ω be the number of categories in the i-th granularity layer, and n be the dimension of the sample features. In this example, l = 3. The number of categories in the first layer, m1, is set to 1, i.e., one class node. The number of categories in the second layer, m2, is 3. The number of categories in the third layer, m3, is 6. i j The weight matrix ω iThe j-th category vector, w i j The weight matrix ω i Disassembled W i The j-th class vector of the matrix, d i j The weight matrix ω i Disassembled W i The j-th category vector of the matrix;

[0067] 3.2) ω i 、W i and D i Substituting these three parameters into the feature selection algorithm of the sparse representation model, we obtain information about W. i D i The loss function J(W) that needs to be optimized is minimized. i D i ):

[0068]

[0069] in Let be the sample matrix of the i-th layer belonging to the j-th class. The corresponding weight vector is ω i j , Sample matrix The number of samples, Sample matrix The corresponding category label, |||| 1,2 It refers to L 1,2 Norms, λ1, and λ2 are nonnegative parameters that control regularization;

[0070] 3.3) Use accelerated gradient descent to minimize the loss function J(W) i D i The solution is iteratively solved until J(W) is reached. i D i The convergence is obtained in k iterations. The optimal solution W i (*) ,

[0071] 3.4) The optimal solution W generated by k iterations i (*) , By adding them together, we obtain the optimal weight matrix ω. i (*) and for ω i (*) The element values ​​are sorted, and the first N features are selected as the N features selected for each layer of the tree structure according to the sorting order.

[0072] Step 4: Select an existing logistic regression classifier as the classifier in the hierarchical structure and train it.

[0073] 4.1) Based on the hierarchical tree structure of the training set, select N features selected in step 3.4) at each level, and use these features to form a feature subset of the training set;

[0074] 4.2) An existing logistic regression classifier is selected as the classifier for the hierarchical structure to be trained. This classifier includes two parameters. and Its loss function It is expressed as follows:

[0075]

[0076]

[0077] Where, p j i Let x be the predicted probability that a training sample belongs to class j in the i-th layer. j For sample data belonging to category j, y j For category label values, The feature weights of the sample data belonging to the j-th class of the i-th layer are... This represents the deviation of the j-th class in the i-th layer;

[0078] 4.3) Iteratively update the logistic regression classifier using a subset of the training set via gradient descent. and deviation These two parameters:

[0079] 4.3.1) Selecting an initial point Let the initial iteration number k = 0;

[0080] 4.3.2) Calculate the loss function in this iteration. At point gradient value

[0081] 4.3.3) Given the termination error ε, the gradient value Compare with ε;

[0082] like Then stop the iteration, and the iteration value of this iteration is... The parameters we are looking for. Obtain a trained hierarchical classifier;

[0083] like Then calculate the point In the next iteration value, and Let k = k + 1, and return to step (4b2), where, It is a loss function right partial derivatives At point The value,

[0084] It is a loss function right partial derivatives exist The value,

[0085] α is the set learning step size.

[0086] Step 5: Perform classification prediction on the medical data samples to be classified in the test set and output the results.

[0087] 5.1) Input the test set samples into the trained hierarchical classifier to obtain the probability that the test sample belongs to the j-th class of the i-th layer of the hierarchical structure.

[0088]

[0089] Where T represents the pair Transpose of; These are the feature weights of the test sample data belonging to the j-th class in the i-th layer. This represents the bias of the test sample data belonging to the j-th class of the i-th layer. These two values ​​are different from those obtained from training the training samples. and equal;

[0090] 5.2) Select the first k pre-selected classes in each layer according to the class probability value, until the first k pre-selected classes in the leaf node layer of the lowest level of the tree hierarchy are obtained;

[0091] 5.3) Use an SVM classifier to determine the final classification from the k pre-selected classes in the leaf layer:

[0092] 5.3.1) Extract all samples belonging to these k categories from the training set subset, merge them into a new training set X′, and use the training set X′ to train the SVM classifier model;

[0093] 5.3.2) Input the test sample back into the trained SVM classifier model and output the final category of the test sample.

[0094] The effects of the present invention will be further explained below with reference to simulation experiments.

[0095] 1. Simulation conditions:

[0096] The hardware platform for the simulation experiment is: a Ryzen 7 4800H CPU with a clock speed of 2.9GHz and 8GB of memory.

[0097] The software platform for the simulation experiment is: Windows 10 operating system, MATLAB 2019a and Python 3.7.

[0098] The simulation data is based on a medical dataset of fever of unknown origin, which is derived from the clinical data and etiological examination results of 564 patients with fever of unknown origin admitted to the First Affiliated Hospital of Xi'an Jiaotong University. Each sample includes pathological data and the doctor's diagnosis of the cause of fever. The pathological data includes medical characteristics such as clinical symptoms, epidemiological history, past medical history, and laboratory test results.

[0099] 2. Simulation content and results:

[0100] Under the above simulation conditions, the clinical data and etiological examination results of 564 patients with fever of unknown origin were stratified and classified using the present invention and three existing methods, TDLR, CSCHIC and HCMP, respectively. The results were compared with the 16 categories of fever etiology diagnosed by doctors, and the accuracy, stratification FH value and tree-induced loss TIE were calculated. The results are shown in Table 1.

[0101] Table 1. Performance comparison of the method of the present invention with other methods.

[0102] method Accuracy Stratified FH value TIE Method of the present invention 75.58％ 86.68％ 45.2 TDLR 67.00％ 82.55％ 62 CSCHIC 66.00％ 81.46％ 64.8 HCMP 70.13％ 83.99％ 58

[0103] As can be seen from the table above, the accuracy, FH and TIE performance of the data detected by the method of the present invention are all superior to the existing detection methods, which proves the correctness and effectiveness of the present invention.

[0104] The parts of this invention not described in detail are common knowledge to those skilled in the art.

[0105] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, those skilled in the art, after understanding the content and principle of the present invention, may make various modifications and changes in form and detail without departing from the principle and structure of the present invention. However, these modifications and changes based on the concept of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A hierarchical classification method for high-dimensional biomedical data based on multi-path feature selection, characterized in that, Includes the following steps: (1) Obtain publicly available high-dimensional biomedical data or medical datasets compiled with data from cooperating hospitals, perform preprocessing such as missing value imputation, encoding and standardization in sequence to form standard data, and divide it into training set and test set in a 9:1 ratio; (2) Form a tree-like hierarchical structure for the dataset based on the medical etiology classification system; (3) Based on L 1,2 Norm regularization performs hierarchical feature selection at each level of the tree-like hierarchical structure: (3a) Let ω be the weight matrix of each granularity layer. i For its two components W i With D i sum: ω i j =w i j +d i j Where i = 1, 2, ..., l, l is the level of the tree structure, and j = 1, 2, ..., m i m i Let ω be the number of categories in the i-th granular layer, n be the dimension of the sample features, and ω be the number of categories in the granular layer. i j The weight matrix ω i The j-th category vector, w i j The weight matrix ω i Disassembled W i The j-th class vector of the matrix, d i j The weight matrix ω i Disassembled W i The j-th category vector of the matrix; (3b) ω i 、W i and D i Substituting these three parameters into the feature selection algorithm of the sparse representation model, we obtain information about W. i D i The loss function J(W) that needs to be optimized is minimized. i D i ): in Let be the sample matrix of the i-th layer belonging to the j-th class. The corresponding weight vector is ω i j , Sample matrix The number of samples, Sample matrix The corresponding category label, || || 1,2 It refers to L 1,2 Norms, λ1, and λ2 are nonnegative parameters that control regularization; (3c) Use accelerated gradient descent to minimize the loss function J(W) i D i The solution is iteratively solved until J(W) is reached. i D i The convergence is obtained in k iterations. optimal solution (3d) The optimal solution generated by k iterations By adding them together, we obtain the optimal weight matrix ω. i (*) and for ω i (*) The element values ​​are sorted, and the first N features are selected as the N features selected for each layer of the tree structure according to the sorting order. (4) Select an existing logistic regression classifier as the classifier in the hierarchical structure and train it. (4a) Based on its tree-like hierarchical structure, select N features selected in step (3d) at each level of the training set, and use these features to form a feature subset of the training set; (4b) Iteratively update the two parameters in the logistic regression classifier using a subset of the training set via gradient descent. and deviation Until its loss function converges, a well-trained logistic regression classifier model is obtained, where... The feature weights of the sample data belonging to the j-th class of the i-th layer are... This represents the deviation of the j-th class in the i-th layer; (5) Perform classification prediction on the medical data samples to be classified in the test set and output the results: (5a) Input the test set samples into the trained hierarchical classifier to obtain the probability that the test sample belongs to the j-th class of the i-th layer of the hierarchical structure. Where T represents the pair Transpose of; These are the feature weights of the test sample data belonging to the j-th class in the i-th layer. This represents the bias of the test sample data belonging to the j-th class of the i-th layer. These two values ​​are different from those obtained from training the training samples. and equal; (5b) Select the first k pre-selected classes in each layer according to the class probability value, until the first k pre-selected classes in the leaf node layer of the lowest layer of the tree hierarchy are obtained; (5c) Use an SVM classifier in the k pre-selected classes of the leaf layer to output the final class of the test sample.

2. The method according to claim 1, characterized in that: In step (1), the data undergoes preprocessing including missing value imputation, encoding, and standardization, as follows: The missing value imputation method involves using KNN interpolation to imput continuous missing data, and using the mode of all data for the data item to imput discrete missing data. The encoding process involves selecting class features that have no magnitude significance from the discrete data after missing value imputation, and then using numerical or vectorized operations to convert them into 0-1 codes and one-hot codes that can be recognized by computers. This 0-1 encoding uses binary variables 0 and 1 to represent the features of each sample across two categories; This one-hot encoding adds each category with more than three categories as a new feature to the dataset; The standardization process involves normalizing the continuous data after missing value imputation, ensuring it is distributed within the [0,1] interval, resulting in preprocessed standardized data x′: Where x represents the original continuous data, MIN represents the minimum value of the data item containing x, and MAX represents the maximum value of the data item containing x.

3. The method according to claim 1, characterized in that: The tree-like hierarchical structure formed in step (2) is a triangular tree-like structure that includes multiple levels of category clusters, wherein: There is only one root node at the top level, which is the parent node of all nodes; The intermediate layer contains multiple category cluster layers, the number of which is determined according to the characteristics of the corresponding medical disease in each medical dataset. Each category cluster layer contains multiple nodes. The bottom layer has multiple nodes, which are called leaf nodes; Each node in each level represents a category, and the leaf nodes are the final classification categories that need to be determined. The entire tree hierarchy runs from the root node to the leaf nodes from top to bottom, corresponding to different etiological categories.

4. The method according to claim 1, characterized in that: In step (4b), an existing logistic regression classifier is selected as the classifier for the hierarchical structure to be trained. This classifier includes two parameters. and Its loss function It is expressed as follows: Where, p j i Let x be the predicted probability that a training sample belongs to class j in the i-th layer. j For sample data belonging to category j, y j For category label values, The feature weights of the sample data belonging to the j-th class of the i-th layer are... This represents the deviation of the i-th layer and j-th class.

5. The method according to claim 1, characterized in that: In step (4b), the gradient descent method is used to solve for the two parameters of the logistic regression classifier model. and deviation The implementation is as follows: (4b1) Selecting the initial point Let the initial iteration number k = 0; (4b2) Calculate the loss function in this iteration. At point gradient value (4b3) Given the termination error ε, the gradient value Compare with ε; like Then stop the iteration, and the iteration value of this iteration is... The parameters we are looking for. Obtain a trained hierarchical classifier; like Then calculate the point In the next iteration value, and Let k = k + 1, and return to step (4b2), where, It is a loss function right partial derivatives At point The value, It is a loss function right partial derivatives exist The value, α is the set learning step size.

6. The method according to claim 1, characterized in that: In step (5c), the SVM classifier is used to determine the final classification among the k pre-selected classes in the leaf layer, as follows: (5c1) Extract all samples belonging to these k categories from the training set subset, merge them into a new training set X′, and use the training set X′ to train the SVM classifier model. (5c2) Input the test sample into the trained SVM classifier model to obtain the final category of the test sample.

Images