A health information processing method based on inter-class mixed decision tree

Abstract

The application provides a health information processing method based on an inter-class mixed decision tree, comprising the following steps: step 1, obtaining a training set X containing user body data; step 2, establishing a target function based on the training set X and obtaining an optimal solution; and step 3, dividing the data into non-intersecting partitions until the nodes become pure or the number of samples is less than a predefined threshold. The application aims to obtain a new inter-class mixed Universum decision tree, combine Universum prior knowledge while avoiding impurity calculation, and thus play a more efficient role in various real-world scenarios (such as disease diagnosis, weather prediction, anomaly detection, etc.). In addition, mixed-induced Universum (MIU) is applied to generate inter-class Universum samples, and MIU is randomly sampled to provide diversity for decision trees in the decision forest, so as to obtain higher disease prediction accuracy.

Description

Technical Field

[0001] This invention belongs to the field of pattern recognition, and in particular relates to a health information processing method based on inter-class hybrid decision trees. Background Technology

[0002] Decision trees, as a long-established supervised learning algorithm, have made significant progress and attracted widespread attention in recent years. Recent research has shown that decision trees outperform neural networks in handling tabular data. In the field of medical disease diagnosis, most data is tabular, giving decision trees an inherent advantage. Furthermore, decision trees can handle challenging problems in finance, healthcare, intrusion detection, weather forecasting, and many other fields. However, existing decision trees face a crucial issue: node splitting criteria. For axially parallel decision trees, a suitable purity metric is key to their splitting criteria. For skewed decision trees, different classifiers or decision functions can be used as splitting criteria. While skewed decision trees smooth the decision hyperplane of axially parallel decision trees, resulting in better performance, they compromise the interpretability of axially parallel decision trees. The combination of Universum learning and decision trees has not been fully explored. This may be because it is difficult to embed the Universum dataset into the binary decision tree construction process. With the Universum dataset introduced, it is difficult to measure the impurity of internal nodes. Furthermore, during tree growth, it is difficult to determine which child node or branch should be assigned to the Universum data.

[0003] In pattern recognition, appropriate prior knowledge helps in obtaining better learning models. Universum prior knowledge assumes that, in addition to the given data, there is usually other domain-specific data that does not belong to any known class. Universum-based algorithms have achieved empirical success in supervised learning, unsupervised learning, and dimensionality reduction. However, to date, Universum has not been applied to the decision tree learning paradigm. There are two main challenges: First, it is difficult to directly embed Universum into decision tree construction because the introduction of Universum data can obfuscate the impurity calculation of decision tree node splits. Second, even if Universum could be embedded in decision tree construction, a poorly designed Universum can degrade the performance of the decision tree. Summary of the Invention

[0004] Purpose of the Invention: Existing decision tree algorithms rely on exhaustive search methods for disease prediction, which is time-consuming and has limited predictive ability. The technical problem this invention aims to solve is to address the shortcomings of existing technologies by providing a hybrid inter-class Universum decision tree algorithm for more effective and efficient disease diagnosis and prediction.

[0005] This invention proposes a novel health information processing method based on inter-class hybrid decision trees to circumvent the problems of existing technologies and apply it to the field of medical diagnosis. Specifically, this invention attempts to select the optimal splitting feature by minimizing the distance between the classification hyperplane and the Universum samples, i.e., by introducing a regularization term. Inspired by geometric mean metric learning, this invention aims to optimize a splitting criterion under the constraint of its diagonalized metric matrix to obtain the discriminative weight of each feature, thereby finding the optimal classification hyperplane and replacing the exhaustive search based on purity metrics in the entire feature space of traditional decision trees. Furthermore, this invention optimizes the objective function to obtain the most discriminative splitting feature to avoid the presence of the Universum dataset from obscuring purity calculations. The Universum samples distributed around the classification hyperplane are hard samples. If hard samples are considered when splitting nodes, a better axis-parallel classification hyperplane can be obtained for each internal node.

[0006] This invention combines geometric mean metric learning and universe learning to obtain a closed-form solution. Specifically, it optimizes a strictly convex objective function to obtain a globally optimal solution, which represents the discriminative weight of each candidate feature at the internal nodes. Intuitively, for the optimal splitting features at the internal nodes: the variance within each class is small, while the distance between samples from different classes is large. Furthermore, universe samples are embedded in the objective function as prior knowledge of the target domain. Specifically, this invention forces universe samples, i.e., hard-to-distinguish samples, to be compactly distributed around the decision surface.

[0007] The technical solution to achieve the purpose of this invention is: a health information processing method based on inter-class hybrid decision trees, the method comprising the following steps:

[0008] Step 1: Obtain the training set X containing user body data;

[0009] Step 2: Establish the objective function based on the training set X and obtain the optimal solution;

[0010] Step 3: Divide the data into disjoint partitions until the nodes become pure or the number of samples is less than a predefined threshold.

[0011] In step 1, the training set X containing user body data is denoted as:

[0012]

[0013] Where x i =[x i1 ,x i2 ,…,x id ] T Let d be the i-th user sample, i = 1, 2, ..., n, and d be x.i Feature dimension, x id ∈R represents the d-th feature value (such as the user's vital signs) of the i-th user, y i ∈{-1,+1} is the label of the i-th user sample, y i A value of -1 indicates that the i-th user is ill, y i A value of 1 indicates that the i-th user is healthy; x'∈U is a Universum sample. In this invention, samples with single quotes all represent artificially generated Universum samples, and U represents the Universum set.

[0014] Step 2 includes:

[0015] Step 2-1: Minimize the Mahalanobis distance d between two user samples x1 and x2. A (x1,x2)=(x1-x2) T A(x1-x2) is equivalent to maximizing Where A is a d×d real symmetric positive definite matrix;

[0016] Maximize the Mahalanobis distance d between the two types of centers A (m1,m2) is equivalent to minimizing Where m1 and m2 represent the class centers of diseased data and healthy data, respectively, and the objective function corresponding to the splitting criterion of each internal node t is... Represented as:

[0017]

[0018] Where tr represents the trace of the matrix. This indicates that A is a positive definite matrix;

[0019] S b S w and S u These are defined as the inter-class covariance matrix, the intra-class covariance matrix, and the Universum intra-class covariance matrix, respectively.

[0020] Step 2-2 aims to obtain a diagonal matrix A = diag(w1,…,w j ,…,w d ), its j-th diagonal element w j The discriminant weight of candidate feature j is given, where feature j represents the j-th individual characteristic of the user and takes values ​​from 1 to d. diag represents a diagonal matrix. Optimizing formula (2) under the constraint of the diagonalized metric matrix yields f(w1,…,w) j ,…,w d );

[0021] Steps 2-3: When the derivative of f(A) is zero, the global optimal solution is obtained, and f(A) with respect to w j Differentiate;

[0022] Steps 2-4 yield the optimal solution;

[0023] In step 2-1, the inter-class covariance matrix S b Intraclass covariance matrix S w and the intraclass covariance matrix S of Universum u As shown below:

[0024] S b =(m1-m2)(m1-m2) T

[0025]

[0026]

[0027] Where c is the class number, m c The center of class C is x, where m1 and m2 represent the class centers of disease data and health data, respectively. i 'and m u ' represents the Universum sample and its class center, respectively, and the superscript T indicates matrix transpose.

[0028] In step 2-2, f(w1,…,w) j ,…,w d As shown below:

[0029]

[0030] Where x ij and m cj Representing user samples x respectively i The j-th eigenvalue and the j-th eigenvalue of the class C mean, x ij 'and m uj 'Represents the Universum sample x respectively i The j-th eigenvalue of ' and the j-th eigenvalue of the Universum class mean.

[0031] In steps 2-3, f(A) is related to w j Differentiation yields:

[0032]

[0033] in This indicates the partial derivative.

[0034] In steps 2-4, setting formula (5) to 0 yields the optimal solution:

[0035]

[0036] Step 3 includes:

[0037] Step 3-1, Optimal splitting feature j of the node * Corresponding to the feature with the highest weight, j * Defined as:

[0038] j * =argmax j w j ,j=1,…,d(7)

[0039] Step 3-2: Split the node according to the optimal splitting feature to obtain two partitions, partition 1 and partition 2. Partition 1 and partition 2 are called child nodes l and r, respectively.

[0040]

[0041] Where st represents the constraint of ..., b represents the node splitting threshold, and X l and X r These represent the training subsets contained in child node l and child node r, respectively.

[0042] Step 3-3: Recursively divide the user data in each node into disjoint partitions using steps 3-1 and 3-2 until the node becomes pure or the number of samples is less than a predefined threshold (generally 10 by default). In this invention, the decision tree stops growing when all user samples in a node belong to the same category (sick or healthy) or when the number of user samples in a node is less than 10.

[0043] The present invention also provides a storage medium storing a computer program or instructions, which, when the computer program or instructions are run, implement the aforementioned health information processing method based on inter-class hybrid decision tree.

[0044] Compared with existing technologies, the significant advantages of this invention are: 1) Decision trees constructed using Universum have not been explored before. This invention proposes a novel inter-class hybrid Universum decision tree, incorporating the Universum principle to assist tree growth; 2) Intuitively, samples distributed near the classification hyperplane are difficult to classify correctly. Simultaneously, Universum samples generated by hybrid-induced Universum are also difficult to classify. If the given intra-class data is far from the classification hyperplane, while the Universum samples are distributed around the hyperplane, the resulting splitting features should be more discriminative. In short, introducing Universum samples provides a regularization term for the positional relationship between different classes in the feature space; 3) The core of ensemble learning is the robustness and diversity of the base learner. Furthermore, decision trees are a common and suitable base learner in ensemble learning. In the inter-class hybrid Universum decision tree, different selections and combinations of Universum samples naturally bring diversity to the decision forest, which is orthogonal to the bagging technique commonly used in random forests. Therefore, by injecting randomness and diversity, i.e., the Universum set, the ensemble model can achieve better generalization performance on unseen datasets. Attached Figure Description

[0045] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0046] Figure 1 This diagram illustrates the correlation between feature discriminative weights and impurity in the inter-class hybrid Universum decision tree algorithm.

[0047] Figure 2a This study analyzes the proportion of Universum data based on the Breast Cancer dataset.

[0048] Figure 2b This is an analysis of the proportion of Universum data based on the Connectionist Bench dataset.

[0049] Figure 2c This study analyzes the proportion of Universum data based on the Molec Biol Promoter dataset.

[0050] Figure 2d This study analyzes the proportion of Universum data based on the Oocytes Trisopterus dataset.

[0051] Figure 2e This is an analysis of the proportion of Universum data based on the Spectf dataset.

[0052] Figure 2f This is an analysis of the proportion of Universum data based on the Sonar dataset.

[0053] Figure 2g This is an analysis of the proportion of Universum data based on the Parkinsons dataset.

[0054] Figure 3 This is a schematic diagram of a Universum generated by hybrid-induced Universum. Detailed Implementation

[0055] This invention describes a health information processing method based on inter-class hybrid decision trees, the method comprising the following steps:

[0056] Step 1, denote the training set (such as user body data) as:

[0057]

[0058] Where x i =[x i1 ,x i2 ,…,x id ] T Let d be the i-th user sample, i = 1, 2, ..., n, and d be x. i Feature dimension, x id ∈R represents a feature value (such as a user's vital signs), y i ∈{-1,+1} is the label of the i-th sample, and in this invention, the value of this label is either sick or healthy; furthermore, x'∈U is a Universum sample, where U represents the Universum set. For example... Figure 3 As shown, in the target domain, in addition to the three known types of data, there are also Universum samples generated by MIU.

[0059] Step 2 requires a brief mention of a theory: minimizing the Mahalanobis distance d between two user samples. A (x1,x2)=(x1-x2) T A(x1-x2) is equivalent to maximizing Where x1 and x2 are two user samples, and A is a d×d real symmetric positive definite matrix.

[0060] Step 3, inspired by Step 2, discovers that maximizing the Mahalanobis distance d between the two types of centers is crucial. A (m1,m2) is equivalent to minimizing Where m1 and m2 represent the class centers of diseased and healthy data, respectively. The objective function corresponding to the splitting criterion of each internal node t is expressed as:

[0061]

[0062] Where tr represents the trace of the matrix. This indicates that A is a positive definite matrix.

[0063] Step 4, S b S w and S u Defined as the inter-class covariance matrix, the intra-class covariance matrix, and the Universum intra-class covariance matrix, respectively:

[0064] S b =(m1-m2)(m1-m2) T ,

[0065]

[0066]

[0067] Where c is the class number, m c It is the center of category C. Specifically, m1 and m2 represent the class centers of disease data and health data, respectively, x i 'and m u 'represents the Universum sample and its class center, respectively.

[0068] Step 5, the goal is to obtain a diagonal matrix A = diag(w1,…,w j ,…,w d ), its j-th diagonal element w j Let represent the discriminative weight of candidate feature j, where feature j represents the j-th individual characteristic of a user, with values ​​ranging from 1 to d, and diag represents a diagonal matrix. Furthermore, the formula in step 3 is optimized under the constraint of the diagonalized metric matrix, and the formula is further rewritten as:

[0069]

[0070] Where x ij and m cj Representing sample x respectively i The j-th feature value of the mean of category C, x ij 'and m uj 'Represents the Universum sample x respectively i The j-th eigenvalue of ' and the j-th eigenvalue of the Universum class mean.

[0071] Step 6: Since the formula in Step 3 is a strictly convex optimization problem, the global optimal solution is obtained when the derivative of f(A) is zero. f(A) with respect to w j Differentiation yields:

[0072]

[0073] Step 7: Set the expression in Step 6 to 0 to obtain the optimal solution.

[0074]

[0075] Step 8: In Step 7, the numerator represents the distance between different classes on feature j, and the denominator is a linear combination of the dispersion of intra-class data and Universum data. The formula obtained in Step 7 is as expected: if the two class centers are far apart, and the distribution within each class (not only intra-class data but also Universum data) is more compact, then feature j has stronger discriminative power and naturally plays a more important role in node splitting. The optimal splitting feature j for a node. * Corresponding to the feature with the highest weight, j * Defined as:

[0076] j * =argmax j w j ,j=1,…,d.

[0077] Step 9: Split the node according to the optimal splitting feature to obtain two partitions, partition 1 and partition 2, which are called child nodes l and r, respectively.

[0078]

[0079]

[0080] Where st represents the constraint of ..., b represents the node splitting threshold, and X l and X r These represent the training subsets contained in the corresponding child nodes.

[0081] Step 10: In this embodiment, steps 8 and 9 are used to recursively divide the user data in each child node into disjoint partitions until the node becomes pure or the number of samples is less than a predefined threshold. In this invention, the decision tree stops growing when all user samples in a node belong to the same category (sick or healthy) or when the number of user samples in a node is less than 10.

[0082] Next, the invention will be further verified and illustrated through experiments.

[0083] The training dataset in this embodiment comes from UCI and OpenML, mainly containing data on diseases such as breast cancer and Parkinson's disease, as well as data from other fields. Furthermore, the experimental data underwent preprocessing. In each internal node of the decision tree, selection... There are 10 candidate features, where d is the feature dimension, i.e., the number of user vital signs. The minleaf parameter is set to 10. Furthermore, given N1 diseased samples and N2 healthy samples, 0.4 × N1 × N2 Universum samples are sampled during the inter-class mixture Universum decision tree growth process. All ensemble models contain 20 base learners.

[0084] This embodiment uses 10-fold cross-validation. Specifically, each dataset is divided into ten parts: nine parts are used for training and one part for testing. The entire 10-fold cross-validation process is repeated ten times, and the average value is recorded to obtain fairer disease diagnosis results.

[0085] First, we investigate the impact of the proportion of Universum samples introduced on the model's disease diagnosis capabilities. The Universum set should contain N1×N2 samples, which is significantly larger than the original dataset. While the introduction of Universum samples brings randomness and diversity to the inter-class mixture Universum decision tree, excessive injection of Universum samples can lead to an imbalance between the original and Universum data, thus negatively impacting the model. Figure 1 This paper analyzes the impact of the proportion of Universe samples on model performance when selecting the optimal splitting feature for internal nodes. Specifically, different proportions of Universe samples were set, and the average classification accuracy of random forests for 21 datasets under different Universes was calculated. Figure 1 The horizontal axis represents the proportion of Universe samples selected from the Universe set U, ranging from 0 to 1. The vertical axis represents the average accuracy of BMURF on 21 datasets. A coordinate x = 0 indicates that the proportion of Universe samples added during tree growth is 0. As the proportion increases, the average accuracy gradually increases, reaching a peak at x = 0.4. After the peak, the accuracy decreases slightly. Introducing too many Universe samples will lead to an imbalance between the original data and the Universe data, which will negatively impact the model. Therefore, an appropriate Universe parameter needs to be selected; in other experiments, this parameter was set to 0.4. This experiment demonstrates that the proposed health information processing method based on inter-class hybrid decision trees has significant effects in disease diagnosis scenarios.

[0086] In disease diagnosis, the inter-class hybrid Universum decision tree in this embodiment differs from the traditional exhaustive search method. Is there a potential correlation between the most discriminative splitting feature obtained in this embodiment and the reduction in impurity? The weight of each feature is calculated on multiple datasets and sorted in descending order. Then, each feature is used to divide the dataset into two partitions, and the impurity of these two partitions is recorded. For ease of observation, each dataset is divided only once.

[0087] like Figure 2a , Figure 2b , Figure 2c , Figure 2d , Figure 2e , Figure 2f , Figure 2g As shown, the horizontal axis represents the features sorted by weight, and the vertical axis represents the impurity of the two partitions after dividing the dataset using the corresponding features. For example, the coordinate x=1 corresponds to the feature with the highest weight. Generally, partitions divided by features with higher weights have lower impurity. The model in this embodiment can perform disease diagnosis more efficiently.

[0088] In disease diagnosis scenarios, traditional decision tree models have achieved good results; however, these models are time-consuming and their performance needs improvement. This invention proposes a novel decision tree from the perspective of Universum learning—the inter-class hybrid Universum decision tree. Inspired by geometric mean metric learning, it considers the geometric positions of intra-class data and Universum data, thereby obtaining more discriminative splitting features at each internal node. Furthermore, different selections and combinations of Universum samples generated by MIU introduce more randomness and diversity into the decision forest, which helps the decision forest generalize better. The performance of the inter-class hybrid Universum decision tree is comparable to or better than other benchmark decision trees. Experiments in this embodiment confirm the effectiveness of the inter-class hybrid Universum decision tree in disease diagnosis. Moreover, applying the inter-class hybrid Universum decision tree to the diagnosis of diseases such as breast cancer and Parkinson's disease yields even better results. Obtaining splitting nodes based on optimization problems rather than exhaustive search is a direction worth exploring. The inter-class hybrid Universum decision tree algorithm can be used as a reference to address sample imbalance or long-tail problems by augmenting more domain-related samples.

[0089] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the content of the health information processing method based on inter-class hybrid decision trees provided by this invention, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0090] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MUU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0091] This invention provides a health information processing method based on inter-class hybrid decision trees. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A health information processing method based on inter-class hybrid decision trees, comprising the following steps: Step 1: Obtain the training set X containing user body data; Step 2: Establish the objective function based on the training set X and obtain the optimal solution; Step 3: Divide the data into disjoint partitions until the nodes become pure or the number of samples is less than a predefined threshold. In step 1, the training set X containing user body data is denoted as: Where x i =[x i1 ,x i2 ,…,x id ] T Let d be the i-th user sample, i = 1, 2, ..., n, and d be x. i Feature dimension, x id ∈R represents the d-th feature value of the i-th user, y i ∈{-1,+1} is the label of the i-th user sample, y i A value of -1 indicates that the i-th user is ill, y i A value of 1 indicates that the i-th user is healthy; x'∈U is a Universum sample, where U represents the Universum set; Step 2 includes: Step 2-1: Minimize the Mahalanobis distance d between two user samples x1 and x2. A (x1,x2)=(x1-x2) T A(x1-x2) is equivalent to maximizing Where A is a d×d real symmetric positive definite matrix; Maximize the Mahalanobis distance d between the two types of centers A (m1,m2) is equivalent to minimizing Where m1 and m2 represent the class centers of diseased data and healthy data, respectively, and the objective function min for the splitting criterion of each internal node t is... A＞ 0f(A) is represented as: min A＞0 f(A)=tr(A(S w +S u ))+tr(A -1 S b ) (2) Where tr represents the trace of the matrix, and A > 0 indicates that A is a positive definite matrix; S b S w and S u These are defined as the inter-class covariance matrix, the intra-class covariance matrix, and the Universum intra-class covariance matrix, respectively. Step 2-2 aims to obtain a diagonal matrix A = diag(w1,…,w j ,…,w d ), its j-th diagonal element w j The discriminant weight of candidate feature j is given, where feature j represents the j-th individual characteristic of the user and takes values ​​from 1 to d. diag represents a diagonal matrix. Optimizing formula (2) under the constraint of the diagonalized metric matrix yields f(w1,…,w) j ,…,w d ); Steps 2-3: When the derivative of f(A) is zero, the global optimal solution is obtained, and f(A) with respect to w j Differentiate; Steps 2-4 yield the optimal solution; In step 2-1, the inter-class covariance matrix S b Intraclass covariance matrix S w and the intraclass covariance matrix S of Universum u As shown below: S b =(m1-m2)(m1-m2) T Where c is the class number, m c The center of class C is x, where m1 and m2 represent the class centers of disease data and health data, respectively. i 'and m u ' represents the Universum sample and its class center, respectively, and the superscript T indicates matrix transpose; In step 2-2, f(w1,…,w) j ,…,w d As shown below: Where x ij and m cj Representing user samples x respectively i The j-th eigenvalue and the j-th eigenvalue of the class C mean, x ij 'and m uj 'Represents the Universum sample x respectively i The j-th eigenvalue of ' and the j-th eigenvalue of the Universum class mean; In steps 2-3, f(A) is related to w j Differentiation yields: in This indicates the partial derivative; In steps 2-4, setting formula (5) to 0 yields the optimal solution: Step 3 includes: Step 3-1, Optimal splitting feature j of the node * Corresponding to the feature with the highest weight, j * Defined as: j * =argmax j w j ,j=1,…,d (7) Step 3-2: Split the node according to the optimal splitting feature to obtain two partitions, partition 1 and partition 2. Partition 1 and partition 2 are called child nodes l and r, respectively. partition 1: partition 2: Where st represents the constraint of ..., b represents the node splitting threshold, and X l and X r These represent the training subsets contained in child node l and child node r, respectively. Step 3-3: Recursively divide the user data in each node into disjoint partitions using steps 3-1 and 3-2 until the node becomes pure or the number of samples is less than a predefined threshold.

2. A storage medium, characterized in that, It stores a computer program or instructions that, when executed, implement the method as described in claim 1.

Images