Classification method and device and storage medium
By calculating the classification reliability of decision trees and the node purity of leaf nodes in the random forest model, and dynamically adjusting the voting weight of each decision tree, the problem of neglecting the reliability differences of decision trees in existing technologies is solved, thus improving the classification accuracy of the model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-26
- Publication Date
- 2026-04-07
AI Technical Summary
In the random forest model, each decision tree has the same voting weight, ignoring the differences in the reliability of the decision trees, which leads to a decrease in classification accuracy.
By obtaining the classification reliability of the decision tree and the node purity of the leaf nodes, the voting weight of each decision tree is calculated, and the final classification label is determined based on these weights.
This improves the classification accuracy of the random forest model and avoids the situation where unreliable decision trees drag down the overall accuracy.
Smart Images
Figure CN121808577A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of artificial intelligence and machine learning technology, and in particular to a classification method, device and storage medium. Background Technology
[0002] In most classification scenarios, such as financial risk control, medical diagnosis, or user behavior prediction, the random forest model is a widely used classification model. It is more balanced in terms of ease of use and interpretability compared to black-box models like neural networks.
[0003] In related technologies, random forests consist of multiple decision trees, each of which determines the final classification label of a sample through a voting mechanism. However, each decision tree votes with the same weight, ignoring the local reliability differences between different decision trees. Decision trees with poor reliability have the same voting weight as those with high reliability, leading to a decrease in the classification accuracy of the random forest model. Summary of the Invention
[0004] This invention provides a classification method, device, and storage medium to improve the classification accuracy of random forest models.
[0005] On the one hand, embodiments of this application provide a classification method, which includes: Based on multiple entity features of the entity to be classified, candidate classification labels are obtained from the outputs of multiple decision trees; the entity features are used to represent the attribute features or behavioral features of the entity to be classified. For any decision tree, obtain the node purity of the leaf nodes that output the candidate classification label during the training phase, and obtain the classification reliability of the decision tree during the validation phase; the node purity characterizes the degree of class concentration of the leaf nodes; the classification reliability characterizes the classification accuracy of the decision tree. Based on the classification reliability of the decision tree and the node purity of the leaf nodes, the voting weight of the decision tree for the entity to be classified is obtained; The classification label of the entity to be classified is determined based on the candidate classification labels output by each decision tree and the voting weight of each decision tree for the entity to be classified.
[0006] Optionally, the plurality of decision trees are obtained by means of: Obtain the historical split gain of each entity feature of multiple training entities as a split feature in the initial stage; The initial decision tree is trained based on multiple first training entities in the initial stage, and the reference gain of each entity feature of the multiple first training entities is obtained as a splitting feature in the initial stage; the reference gain is determined based on the historical splitting gain as a splitting feature and the number of times it has been used as a splitting feature. The decision tree is trained based on multiple second training entities during the training phase. For any entity feature, Based on the reference gain of the entity feature and the training split gain during the training phase, the split value score of the entity feature during the training phase is determined. The decision tree is obtained by training based on the split value score of each entity feature.
[0007] Optionally, the reference gain is determined based on historical splitting gain as a splitting feature and the number of splitting events as a splitting feature, including: The entity features are summed as gain values for each split feature in the initial stage to obtain the historical split gain; Based on the historical split gain and the number of times the entity feature is used as a split feature in the initial stage, a reference gain for the entity feature is determined. Based on the reference gain of the entity feature, the training split gain of the entity feature during the training phase, and the buffer parameter, the split value score of the entity feature during the training phase is determined; the buffer parameter is used to ensure the effectiveness of the split value score.
[0008] Optionally, obtaining the node purity of the leaf nodes that output the candidate classification labels in the decision tree during the training phase includes: Obtain the true label of each training entity in the leaf node of the decision tree during the training phase; The node purity of a leaf node is obtained by calculating the proportion of the multiple training entities with the same real label that have the largest number of occurrences among all training entities in that leaf node.
[0009] Optionally, obtaining the classification reliability determined by the decision tree during the validation phase includes: Based on multiple entity features of each verification entity, obtain the verification label for each verification entity; Count the number of correctly verified entities whose verification labels and real labels are the same; The classification reliability of the decision tree in the verification phase is determined based on the number of correct verifications and the total number of verification entities in the decision tree.
[0010] Optionally, obtaining the voting weight of the decision tree for the entity to be classified based on the classification reliability of the decision tree and the node purity of the leaf nodes includes: The voting weights of the decision tree are determined based on the classification reliability of the decision tree, the node purity of the leaf nodes, and the stability coefficient; the stability coefficient is used to balance the classification reliability of the decision tree with the node purity of the leaf nodes of the decision tree.
[0011] Optionally, determining the classification label of the entity to be classified based on the candidate classification labels output by each decision tree and the voting weight of each decision tree for the entity to be classified includes: The voting weights of multiple decision trees that output the same candidate classification label are summed to obtain the voting score; The candidate category label with the highest voting score is used as the category label for the entity to be classified.
[0012] On one hand, embodiments of this application provide a sorting device, which includes: The prediction acquisition module is used to obtain candidate classification labels output by multiple decision trees based on multiple entity features of the entity to be classified; the entity features are used to represent the attribute features or behavioral features of the entity to be classified. The training acquisition module is used to acquire, for any decision tree, the node purity of the leaf nodes that output the candidate classification label during the training phase, and the classification reliability of the decision tree during the validation phase; the node purity characterizes the degree of class concentration of the leaf nodes; the classification reliability characterizes the classification accuracy of the decision tree. The weight determination module is used to obtain the voting weight of the decision tree for the entity to be classified based on the classification reliability of the decision tree and the node purity of the leaf nodes. The label determination module is used to determine the classification label of the entity to be classified based on the candidate classification labels output by each decision tree and the voting weight of each decision tree for the entity to be classified.
[0013] Optionally, the training acquisition module is specifically used for: Obtain the historical split gain of each entity feature of multiple training entities as a split feature in the initial stage; The initial decision tree is trained based on multiple first training entities in the initial stage, and the reference gain of each entity feature of the multiple first training entities is obtained as a splitting feature in the initial stage; the reference gain is determined based on the historical splitting gain as a splitting feature and the number of times it has been used as a splitting feature. The decision tree is trained based on multiple second training entities during the training phase. For any entity feature, Based on the reference gain of the entity feature and the training split gain during the training phase, the split value score of the entity feature during the training phase is determined. The decision tree is obtained by training based on the split value score of each entity feature.
[0014] Optionally, the training acquisition module is specifically used for: The entity features are summed as gain values for each split feature in the initial stage to obtain the historical split gain; Based on the historical split gain and the number of times the entity feature is used as a split feature in the initial stage, a reference gain for the entity feature is determined. Based on the reference gain of the entity feature, the training split gain of the entity feature during the training phase, and the buffer parameter, the split value score of the entity feature during the training phase is determined; the buffer parameter is used to ensure the effectiveness of the split value score.
[0015] Optionally, the training acquisition module is specifically used for: Obtain the true label of each training entity in the leaf node of the decision tree during the training phase; The node purity of a leaf node is obtained by calculating the proportion of the multiple training entities with the same real label that have the largest number of occurrences among all training entities in that leaf node.
[0016] Optionally, the training acquisition module is specifically used for: Based on multiple entity features of each verification entity, obtain the verification label for each verification entity; Count the number of correctly verified entities whose verification labels and real labels are the same; The classification reliability of the decision tree in the verification phase is determined based on the number of correct verifications and the total number of verification entities in the decision tree.
[0017] Optionally, the weight determination module is specifically used for: The voting weights of the decision tree are determined based on the classification reliability of the decision tree, the node purity of the leaf nodes, and the stability coefficient; the stability coefficient is used to balance the classification reliability of the decision tree with the node purity of the leaf nodes of the decision tree.
[0018] Optionally, the label determination module is specifically used for: The voting weights of multiple decision trees that output the same candidate classification label are summed to obtain the voting score; The candidate category label with the highest voting score is used as the category label for the entity to be classified.
[0019] On one hand, embodiments of this application provide a computer device, including: Memory, used to store program instructions; The processor is used to call the program instructions stored in the memory and execute the steps of the above classification method according to the obtained program.
[0020] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer to perform the steps of the above-described classification method.
[0021] On one hand, embodiments of this application provide a computer program product, including a computer program stored on a computer-readable storage medium, the computer program including program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described classification method.
[0022] In this embodiment, candidate classification labels are obtained from multiple decision trees based on multiple entity features of the entity to be classified. For any decision tree, the node purity of the leaf nodes that output candidate classification labels during the training phase and the classification reliability of the decision tree during the verification phase are obtained. Based on the classification reliability and node purity of the leaf nodes, the voting weights of the decision trees for the entities to be classified are obtained. Based on the candidate classification labels output by each decision tree and the voting weights of each decision tree for the entities to be classified, the classification label of the entity to be classified is determined. Based on the degree of category concentration of the leaf nodes represented by node purity and the classification accuracy of the decision trees represented by classification reliability, a voting weight is determined for each decision tree. This avoids the situation in the prior art where decision trees with the same voting weight lead to decision trees with low classification accuracy dragging down the overall accuracy of the random forest model, thereby improving the classification accuracy of the random forest model. Attached Figure Description
[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0024] Figure 1 A schematic diagram of a system architecture provided in an embodiment of this application; Figure 2 A flowchart illustrating a classification method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a sorting device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0025] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.
[0026] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.
[0027] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.
[0028] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.
[0029] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.
[0030] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
[0031] The following is an explanation of the technical terms used in the embodiments of this application: Random Forest (RF): An ensemble classification algorithm consisting of multiple decision trees.
[0032] Decision Tree: A tree-structured classification model based on feature splitting.
[0033] Bootstrap sampling: Samples are drawn from the original data with replacement to train each decision tree.
[0034] Gini Impurity: A metric that measures the degree of heterogeneity in data.
[0035] The following is a brief introduction to the system architecture diagram applicable to the technical solutions of the embodiments of this application. It should be noted that the process described below is only for illustrating the embodiments of this application and is not intended to limit the scope of the application.
[0036] refer to Figure 1 This is a system architecture diagram applicable to the embodiments of this application. The system architecture includes at least terminal device 101 and server 102. The number of terminal devices 101 can be one or more, and the number of servers 102 can also be one or more. This application does not specifically limit the number of terminal devices 101 and servers 102.
[0037] Terminal device 101 is pre-installed with applications that have categorization functions. These applications can be client applications, web applications, mini-program applications, etc. Terminal device 101 can be a smartphone, tablet, laptop, desktop computer, smart home appliance, smart voice interaction device, smart in-vehicle device, etc., but is not limited to these.
[0038] Server 102 is the backend server of the application. Server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, but it is not limited to these.
[0039] It should be noted that the method in this embodiment can be executed by the terminal device 101 or the server 102 alone, or by the terminal device 101 and the server 102 together.
[0040] In this embodiment, the terminal device 101 and the server 102 can communicate directly or indirectly through one or more networks. The network can be a wired network or a wireless network; for example, the wireless network can be a mobile cellular network or a Wireless-Fidelity (WIFI) network, or other possible networks. This embodiment does not limit the types of networks used.
[0041] The following is based on Figure 1The system architecture diagram shown illustrates the flowchart of a classification method provided in this application embodiment. This method's flowchart can be... Figure 1 The execution can be performed by the terminal device 101 shown, or by the server 102, or by interaction between the terminal device 101 and the server 102, as shown. Figure 2 As shown, it includes the following steps: Step 201: Based on multiple entity features of the entity to be classified, obtain candidate classification labels output by multiple decision trees respectively.
[0042] Specifically, the multiple decision trees are decision trees in a pre-trained random forest. An entity sample set is divided into a training set, a validation set, and a prediction set according to a certain ratio. This ratio can be 7:1.5:1.5 (training:validation:prediction) or 8:1:1. The specific ratio can be set based on actual needs, and this application does not impose a specific limitation. The entity to be classified is a sample entity in the prediction set.
[0043] In this embodiment, entity features are used to represent the attribute features or behavioral features of the entity to be classified. For example, if the entity to be classified is a user, the attribute features can be user gender, age, occupation, savings, etc., and the behavioral features can be whether the user clicks on advertising SMS messages, listens to advertising phone calls, or has purchased financial products, etc. The candidate classification labels can be whether the user will purchase fixed deposits or not. If the entity to be classified is a device, the attribute features can be operating temperature, operating voltage, lifespan, running time, etc., and the behavioral features can be whether the device has been maintained recently and the number of maintenance sessions, etc. The candidate classification labels can be healthy, slightly degraded, moderately degraded, severely degraded, etc.
[0044] In some embodiments, multiple decision trees are obtained by: acquiring the historical split gain of each entity feature of multiple training entities as a splitting feature in the initial stage; training an initial decision tree based on multiple first training entities in the initial stage to obtain the reference gain of each entity feature of multiple first training entities as a splitting feature in the initial stage; the reference gain is determined based on the historical split gain as a splitting feature and the number of times it is used as a splitting feature; training a decision tree based on multiple second training entities in the training stage, and for any entity feature, determining the splitting value score of the entity feature in the training stage based on the reference gain of the entity feature and the training split gain in the training stage; and training to obtain a decision tree based on the splitting value score of each entity feature.
[0045] In some embodiments, the gain values of entity features used as splitting features each time in the initial stage are summed to obtain historical splitting gain; a reference gain of entity features is determined based on the historical splitting gain and the number of times entity features are used as splitting features in the initial stage; and a splitting value score of entity features in the training stage is determined based on the reference gain of entity features, the training splitting gain of entity features in the training stage, and buffer parameters.
[0046] In this embodiment, the buffer parameter is used to ensure the validity of the split value score.
[0047] Specifically, a portion of the data in the training set is used as initial data, and the initial decision tree is trained using the traditional random forest training method. Taking any current node of any initial decision tree as an example, the traditional training method is introduced: The current node's internal sample container contains M first training entities, each with N entity features and a true label. K entity features are randomly selected from the M training entities as candidate entity features. K is preset through hyperparameters.
[0048] For any candidate entity feature, calculate the gain value of its split feature as any current node in the initial decision tree. If the candidate entity feature is a low-cardinality discrete feature, such as gender, then calculate the Gini impurity of the M1 first training entities split into the left child node (male) of the current node and the Gini impurity of the M2 first training entities split into the right child node (female). If the candidate entity feature is a high-cardinality continuous value, then sort the specific values of the non-repeating candidate entity feature from smallest to largest, and take the median of any adjacent specific values as the candidate split value of the candidate entity feature. Calculate the Gini impurity of the M1 first training entities split into the left child node (less than or equal to the candidate split value) of the current node and the Gini impurity of the M2 first training entities split into the right child node (greater than the candidate split value). Perform a weighted sum of the Gini impurities of the left and right child nodes to obtain the Gini impurity of the candidate entity feature split at the candidate split value. For the current node, calculate the ratio of the number of first training entities with multiple true labels of the same class among the M first training entities to the total number of M first training entities. Calculate the Gini impurity of the current node based on multiple ratios. Based on the Gini impurity of the current node and the Gini impurity of the candidate entity feature splitting at the candidate split value, obtain the Gini decrease (i.e., gain value) of the candidate entity feature at the candidate split value. Take the candidate entity feature with the largest Gini decrease as the split feature of the current node.
[0049] The formula for calculating the Gini impurity of the current node is shown in the following formula (1): (1) in, This indicates the Gini impurity of the current node; This indicates the number of the first training entities contained in the current node; This represents the number of true positive labels among the M first training entities; This represents the number of true negative labels among the M first training entities.
[0050] The formula for calculating the Gini impurity of the left child node formed after splitting the candidate training entity at the candidate splitting value is shown in the following formula (2): (2) in, This represents the Gini impurity of the left child node of the current node; This indicates the number of the first training entities contained in the left child node; express The number of entities with positive true labels in the first training entity; Show The number of true negative labels in the first training entity.
[0051] The formula for calculating the Gini impurity of the formed right child node is shown in the following formula (3): (3) in, This represents the Gini impurity of the right child node of the current node; This indicates the number of the first training entities contained in the right child node; express The number of entities with positive true labels in the first training entity; Show The number of entities with negative true labels in the first training entity; .
[0052] The formula for calculating the Gini impurity of the candidate entity feature as a splitting feature (i.e., the candidate entity feature splits at the candidate splitting value) is shown in the following formula (4): (4) in, This represents the Gini impurity of the candidate entity features as splitting features; This indicates the number of the first training entities contained in the current node; This indicates the number of the first training entities contained in the left child node; This represents the Gini impurity of the left child node of the current node; This indicates the number of the first training entities contained in the right child node; The Gini impurity of the right child node of the current node.
[0053] The formula for calculating the Gini decrease (i.e., gain value) of the candidate entity feature as a splitting feature (i.e., the candidate entity feature splits at the candidate splitting value) is shown in the following formula (5): (5) in, This represents the gain value (i.e., Gini decrease) of entity feature f as a classification feature. This indicates the Gini impurity of the current node; This represents the Gini impurity of the two child nodes of the current node.
[0054] It should be noted that the above formula is presented from the perspective of a binary classification task. If the classification task is a multi-class task, any current node will still only split into two child nodes: the left child node and the right child node, but the number of real labels will be greater than 2.
[0055] In the initial stage, any entity feature of the first training entity may be selected multiple times as the splitting feature of any current node in any decision tree. A list gain_records[f] is set up. Whenever an entity feature is selected as a splitting feature in the initial stage, its gain value when it is selected as a splitting feature is added to the list gain_records[f]. The number of times each entity feature is selected as a splitting feature and the total Gini impurity (i.e., historical splitting gain) as a splitting feature are calculated based on the list. Based on the number of times and the total Gini impurity, the reference gain of the entity feature at the initial node is calculated. The specific calculation formula is shown in the following formula (6): (6) in, This represents the reference gain of entity features as splitting features in the initial stage; This represents the historical splitting gain of entity features as splitting features in the initial stage; This indicates the number of times an entity feature is used as a splitting feature in the initial stage.
[0056] For example, using 4000 samples of the first training entities to train 10 initial decision trees in the initial stage, the four entity features of the first training entities are: account balance, whether the marketing SMS was clicked, number of times contacted, and employment index. The number of times each entity feature is used as a splitting feature and the total Gini coefficient decrease in the initial stage are shown in the table below: Table 1
[0057] In Table 1, the reference gain for each entity feature is saved and used during the training phase for selecting classification features.
[0058] In this embodiment, the reference gain is obtained by directly dividing the total Gini decrease by the total number of splits of the entity feature at the initial node. Compared with normalization methods such as Z-score or min-max, this method can maintain the discriminative ability of a single decision tree and alleviate the overfitting phenomenon caused by the monopoly of split features by high cardinality continuous features in traditional training methods, which fails to observe the occasional high gain of noisy features. Secondly, the random forest in the traditional training method does not have a historical performance memory mechanism, so it cannot be used to select split features in the prediction stage based on the historical average gain. Therefore, in this technical solution, a portion of the training set data is used specifically for training with the traditional training method to record the historical average gain (i.e., reference gain) of each entity feature, thereby improving the accuracy of random forest classification or prediction.
[0059] Furthermore, another part of the data in the training set (multiple second training entities) is used to train the decision tree. For any decision tree, when selecting the splitting feature of any current node in any decision tree, the entity feature with the largest Gini decrease is no longer selected in the traditional training method. Instead, the splitting value score of the entity feature is calculated based on the Gini decrease calculated in this instance, as well as the reference gain and buffer parameters. The entity feature with the largest splitting value score is then used as the splitting feature of the current node. The formula for calculating the splitting value score of the entity feature is shown in the following formula (7): (7) in, Represent entity features f The value of division; This represents the entity features calculated in this experiment. f The gain value (i.e., Gini impurity); Represent entity features f Reference gain; represents the buffer parameter, which is a very small value greater than zero, used to ensure that the denominator in formula (7) is not zero.
[0060] It should be noted that after selecting the splitting feature, each training entity (the first training entity in the initial stage and the second training entity of the training node) that meets the splitting feature value, as well as the specific values of multiple entity features, are assigned to the internal sample container of the corresponding child node. When it is determined that the number of training entities in the internal sample container of a certain child node is less than the preset minimum number of splitting samples, or the candidate entity features obtained from the internal sample container of that child node can no longer bring effective gain values, that is, it is impossible to continue splitting, then the node is marked as the leaf node of the initial decision tree. When each decision tree can no longer split and the decision tree depth reaches the upper limit, multiple pre-trained decision trees are obtained.
[0061] In this embodiment, considering that in traditional random forest training methods, regardless of whether it is for low-cardinality discrete features or high-cardinality continuous features, the Gini descent is calculated to determine the splitting feature. However, the magnitude of the Gini descent depends not only on the discriminative power of the entity feature but also on the cardinality of the values. For example, high-cardinality continuous features have a large number of candidate splitting values to choose from, and the two child nodes formed after splitting high-cardinality continuous features form internal sample containers with extreme distributions. For example, the internal sample container of a certain child node may contain only training entities with positive true labels, resulting in a significant reduction in the Gini impurity of the splitting feature, thus resulting in a large Gini descent. On the other hand, low-cardinality discrete features only contain a limited number of splitting methods, making it difficult to compete with high-cardinality continuous features and affecting fairness. In this application, the historical splitting gain and the number of historical splits (i.e., the number of times it was used as a splitting feature in the initial stage) of each entity feature are used to calculate the average historical gain as a reference gain. This weakens the problem of excessively large or small single Gini descent of entity features, enabling low-cardinality discrete features to compete relatively fairly with high-cardinality continuous features and improving the robustness of entity classification.
[0062] For example, if entity feature A is a low-cardinality discrete feature with a reference gain of 0.08, and its current Gini decrease is 0.087; entity feature B is a high-cardinality continuous feature with a reference gain of 0.25, and its current Gini decrease is 0.1633. If the splitting feature is selected based on the magnitude of the Gini decrease according to the traditional training method, entity feature A cannot compete with entity feature B. However, through the calculation of formula (7), the current splitting value of entity feature A is 1.09, and the splitting value of entity feature B is 0.65 < 1.09. Therefore, the reference gain provides a relatively fair competitive environment for low-cardinality discrete features with relatively small historical average gains.
[0063] Step 202: For any decision tree, obtain the node purity of the leaf nodes that output candidate classification labels during the training phase, and obtain the classification reliability of the decision tree during the validation phase.
[0064] In the embodiments of this application, node purity characterizes the degree of class concentration of leaf nodes; classification reliability characterizes the classification accuracy of the decision tree.
[0065] In some embodiments, the true label of each training entity in the leaf node of the decision tree during the training phase is obtained; the proportion of the multiple training entities with the same true label with the largest number of occurrences in all training entities in the leaf node is calculated to obtain the node purity of the leaf node.
[0066] Specifically, after obtaining multiple pre-trained decision trees, for each leaf node of each decision tree, the true label of each training entity is obtained from the internal sample container of the leaf node, and the number of training entities with positive true labels and the number of training entities with negative true labels are counted. The maximum number of training entities with more true labels is determined. Based on the maximum number and the total number of training entities in the leaf node, the node purity of the leaf node is determined, and the maximum number of true labels is used as the predicted classification label of the leaf node, that is, the candidate split label output by the leaf node in future prediction.
[0067] The specific formula for calculating node purity is shown in the following formula (8): (8) in, Indicates the purity of a leaf node; This represents the number of training entities with the correct true labels; This represents the number of training entities with negative true labels; This represents the total number of all trained entities in the leaf node.
[0068] It should be noted that the technical solution of this application is also applicable to multi-classification tasks where there are multiple real labels.
[0069] In some embodiments, a verification label for each verification entity is obtained based on multiple entity features of each verification entity; the number of correct verifications for verification entities with the same verification label and true label is counted; and the classification reliability of the decision tree in the verification phase is determined based on the number of correct verifications and the total number of verification entities in the decision tree.
[0070] Specifically, the validation set is input into multiple decision trees for validation. For each validation sample in the validation set, it is divided from the root node of each decision tree through multiple internal child nodes of the decision tree to a certain leaf node. The predicted classification label of the leaf node is the validation label of the decision tree for the validation entity. The number of correct validations of multiple validation entities in the validation set whose validation labels match the true labels of each decision tree is counted. The number of correct validations is divided by the number of all validation entities output to the decision tree to obtain the classification reliability of the decision tree in the validation stage, which represents the classification accuracy of the decision tree. The specific calculation formula for the classification reliability of the decision tree is shown in the following formula (9): (9) in, Indicates the classification reliability of the decision tree; This indicates the number of correct verification labels for multiple verified entities in the decision tree; This indicates the number of validating entities input into the decision tree.
[0071] In this embodiment, the node purity of each leaf node of each decision tree during the training phase and the classification reliability of each decision tree during the verification phase are recorded. Compared with the traditional training method that only outputs classification labels without retaining any internal information of the decision tree or leaf nodes, explicitly recording node purity and classification reliability can provide a data basis for determining the voting weight of each decision tree in the subsequent prediction phase, thereby improving classification accuracy.
[0072] Step 203: Based on the classification reliability of the decision tree and the node purity of the leaf nodes, obtain the voting weights of the decision tree for entities to be classified.
[0073] In some embodiments, the voting weights of the decision tree are determined based on the classification reliability of the decision tree, the node purity of the leaf nodes, and the stability coefficient; the stability coefficient is used to balance the classification reliability of the decision tree with the node purity of the leaf nodes of the decision tree.
[0074] Specifically, the formula for calculating the voting weights of the decision tree is shown in the following formula (10). (10) in, This represents the voting weight of the i-th decision tree; Indicates the classification reliability of the decision tree; Indicates the purity of a leaf node; It is the stability coefficient, and .
[0075] It should be noted that, based on actual verification, It can control the dominant contribution of node purity to voting weight. The lower limit that guarantees classification reliability; Set to 0.8 Setting it to 0.2 can balance the classification accuracy of decision trees and the global stability of random forests in most business scenarios; alternatively, the stability coefficient can be adjusted by grid search or Bayesian optimization on an independent validation set according to the actual needs of the business scenario. This application does not make specific limitations on this.
[0076] Step 204: Based on the candidate classification labels output by each decision tree and the voting weight of each decision tree for the entity to be classified, determine the classification label of the entity to be classified.
[0077] In some embodiments, the voting weights of multiple decision trees that output the same candidate classification label are summed to obtain a voting score; the candidate classification label with the highest voting score is used as the classification label of the entity to be classified.
[0078] Specifically, in the label classification task, for any entity to be classified, multiple candidate classification labels are used to sum the voting weights of multiple decision trees that output the same candidate classification label to obtain a voting score, and the candidate classification label with the highest voting score is used as the classification label of the entity to be classified; in the numerical prediction task, the candidate prediction value output by each decision tree (equivalent to the candidate classification label in the label classification task) is weighted and summed with the voting weight of the corresponding decision tree to obtain the final prediction value.
[0079] For example, if the four candidate classification labels for an entity to be classified are "will buy fixed deposits", "will buy fixed deposits", "will not buy fixed deposits", and "will buy fixed deposits", but the corresponding decision tree voting weights are 1, 0.96, 0.97, and 1.1, then the ratio of "will buy fixed deposits": "will not buy fixed deposits" is 1.96:2.07, and the final classification label for the entity to be classified is "will not buy fixed deposits".
[0080] For another example, if the prediction event is the water flow velocity (in m / s) for the next day, and there are four candidate predicted values of 0.67, 0.98, 0.89, and 0.77, but the corresponding decision tree voting weights are 0.23, 0.3, 0.15, and 0.32, then the final predicted value for the entity to be predicted is... If the value is 0.828, then the predicted final water flow velocity is 0.828 m / s.
[0081] In this embodiment, candidate classification labels are obtained from multiple decision trees based on multiple entity features of the entity to be classified. For any decision tree, the node purity of the leaf nodes that output candidate classification labels during the training phase and the classification reliability of the decision tree during the verification phase are obtained. Based on the classification reliability and node purity of the leaf nodes, the voting weights of the decision trees for the entities to be classified are obtained. Based on the candidate classification labels output by each decision tree and the voting weights of each decision tree for the entities to be classified, the classification label of the entity to be classified is determined. Based on the degree of category concentration of the leaf nodes represented by node purity and the classification accuracy of the decision trees represented by classification reliability, a voting weight is determined for each decision tree. This avoids the situation in the prior art where decision trees with the same voting weight lead to decision trees with low classification accuracy dragging down the overall accuracy of the random forest model, thereby improving the classification accuracy of the random forest model.
[0082] Based on the same technical concept, this application provides a schematic diagram of the structure of a sorting device, such as... Figure 3 As shown, the sorting device 300 includes: The prediction acquisition module 301 is used to obtain candidate classification labels output by multiple decision trees based on multiple entity features of the entity to be classified; the entity features are used to represent the attribute features or behavioral features of the entity to be classified. The training acquisition module 302 is used to acquire, for any decision tree, the node purity of the leaf nodes that output the candidate classification label during the training phase, and the classification reliability of the decision tree during the verification phase; the node purity characterizes the degree of class concentration of the leaf nodes; the classification reliability characterizes the classification accuracy of the decision tree. The weight determination module 303 is used to obtain the voting weight of the decision tree for the entity to be classified based on the classification reliability of the decision tree and the node purity of the leaf nodes. The label determination module 304 is used to determine the classification label of the entity to be classified based on the candidate classification labels output by each decision tree and the voting weight of each decision tree for the entity to be classified.
[0083] Optionally, the training acquisition module 302 is specifically used for: Obtain the historical split gain of each entity feature of multiple training entities as a split feature in the initial stage; The initial decision tree is trained based on multiple first training entities in the initial stage, and the reference gain of each entity feature of the multiple first training entities is obtained as a splitting feature in the initial stage; the reference gain is determined based on the historical splitting gain as a splitting feature and the number of times it has been used as a splitting feature. The decision tree is trained based on multiple second training entities during the training phase. For any entity feature, Based on the reference gain of the entity feature and the training split gain during the training phase, the split value score of the entity feature during the training phase is determined. The decision tree is obtained by training based on the split value score of each entity feature.
[0084] Optionally, the training acquisition module 302 is specifically used for: The entity features are summed as gain values for each split feature in the initial stage to obtain the historical split gain; Based on the historical split gain and the number of times the entity feature is used as a split feature in the initial stage, a reference gain for the entity feature is determined. Based on the reference gain of the entity feature, the training split gain of the entity feature during the training phase, and the buffer parameter, the split value score of the entity feature during the training phase is determined; the buffer parameter is used to ensure the effectiveness of the split value score.
[0085] Optionally, the training acquisition module 302 is specifically used for: Obtain the true label of each training entity in the leaf node of the decision tree during the training phase; The node purity of a leaf node is obtained by calculating the proportion of the multiple training entities with the same real label that have the largest number of occurrences among all training entities in that leaf node.
[0086] Optionally, the training acquisition module is specifically used for: Based on multiple entity features of each verification entity, obtain the verification label for each verification entity; Count the number of correctly verified entities whose verification labels and real labels are the same; The classification reliability of the decision tree in the verification phase is determined based on the number of correct verifications and the total number of verification entities in the decision tree.
[0087] Optionally, the weight determination module 303 is specifically used for: The voting weights of the decision tree are determined based on the classification reliability of the decision tree, the node purity of the leaf nodes, and the stability coefficient; the stability coefficient is used to balance the classification reliability of the decision tree with the node purity of the leaf nodes of the decision tree.
[0088] Optionally, the label determination module 304 is specifically used for: The voting weights of multiple decision trees that output the same candidate classification label are summed to obtain the voting score; The candidate category label with the highest voting score is used as the category label for the entity to be classified.
[0089] In this embodiment, candidate classification labels are obtained from multiple decision trees based on multiple entity features of the entity to be classified. For any decision tree, the node purity of the leaf nodes that output candidate classification labels during the training phase and the classification reliability of the decision tree during the verification phase are obtained. Based on the classification reliability and node purity of the leaf nodes, the voting weights of the decision trees for the entities to be classified are obtained. Based on the candidate classification labels output by each decision tree and the voting weights of each decision tree for the entities to be classified, the classification label of the entity to be classified is determined. Based on the degree of category concentration of the leaf nodes represented by node purity and the classification accuracy of the decision trees represented by classification reliability, a voting weight is determined for each decision tree. This avoids the situation in the prior art where decision trees with the same voting weight lead to decision trees with low classification accuracy dragging down the overall accuracy of the random forest model, thereby improving the classification accuracy of the random forest model.
[0090] Based on the same technical concept, embodiments of this application provide a computer device, which can be... Figure 1 The server shown is as follows: Figure 4 As shown, it includes at least one processor 401 and a memory 402 connected to at least one processor. In this embodiment, the specific linking medium between the processor 401 and the memory 402 is not limited. Figure 4 Taking the connection between processor 401 and memory 402 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.
[0091] In this embodiment of the application, the memory 402 stores instructions executed by at least one processor 401. By executing the instructions stored in the memory 402, at least one processor 401 can perform the steps of the above-described classification method.
[0092] The processor 401 is the control center of the computer device, capable of connecting various parts of the computer device via various interfaces and lines. It performs classification by running or executing instructions stored in the memory 402 and accessing data stored in the memory 402. Optionally, the processor 401 may include one or more processing modules. The processor 401 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401. In some embodiments, the processor 401 and the memory 402 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.
[0093] Processor 401 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0094] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 402 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 402 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer device, but is not limited thereto. In the embodiments of this application, memory 402 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.
[0095] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described classification method.
[0096] Based on the same inventive concept, embodiments of this application provide a computer program product, including a computer program stored on a computer-readable storage medium, the computer program including program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described classification method.
[0097] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0098] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0099] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0100] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0101] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A classification method, characterized in that, include: Based on multiple entity features of the entity to be classified, candidate classification labels are obtained from the outputs of multiple decision trees. The entity features are used to represent the attribute features or behavioral features of the entity to be classified; For any decision tree, obtain the node purity of the leaf nodes that output the candidate classification label during the training phase, and obtain the classification reliability of the decision tree during the validation phase; the node purity characterizes the degree of class concentration of the leaf nodes. The classification reliability characterizes the classification accuracy of the decision tree; Based on the classification reliability of the decision tree and the node purity of the leaf nodes, the voting weight of the decision tree for the entity to be classified is obtained; The classification label of the entity to be classified is determined based on the candidate classification labels output by each decision tree and the voting weight of each decision tree for the entity to be classified.
2. The method as described in claim 1, characterized in that, The multiple decision trees were obtained in the following ways: Obtain the historical split gain of each entity feature of multiple training entities as a split feature in the initial stage; The initial decision tree is trained based on multiple first training entities in the initial stage, and the reference gain of each entity feature of the multiple first training entities is obtained as a splitting feature in the initial stage; the reference gain is determined based on the historical splitting gain as a splitting feature and the number of times it has been used as a splitting feature. The decision tree is trained based on multiple second training entities during the training phase. For any entity feature, Based on the reference gain of the entity feature and the training split gain during the training phase, the split value score of the entity feature during the training phase is determined. The decision tree is obtained by training based on the split value score of each entity feature.
3. The method as described in claim 2, characterized in that, The reference gain is determined based on the historical split gain as a splitting feature and the number of splits as a splitting feature, including: The entity features are summed as gain values for each split feature in the initial stage to obtain the historical split gain; Based on the historical split gain and the number of times the entity feature is used as a split feature in the initial stage, a reference gain for the entity feature is determined. Based on the reference gain of the entity feature, the training split gain of the entity feature during the training phase, and the buffer parameter, the split value score of the entity feature during the training phase is determined; the buffer parameter is used to ensure the effectiveness of the split value score.
4. The method as described in claim 1, characterized in that, The step of obtaining the node purity of the leaf nodes that output the candidate classification labels in the decision tree during the training phase includes: Obtain the true label of each training entity in the leaf node of the decision tree during the training phase; The node purity of a leaf node is obtained by calculating the proportion of the multiple training entities with the same real label that have the largest number of occurrences among all training entities in that leaf node.
5. The method as described in claim 1, characterized in that, The step of obtaining the classification reliability determined by the decision tree during the validation phase includes: Based on multiple entity features of each verification entity, obtain the verification label for each verification entity; Count the number of correctly verified entities whose verification labels and real labels are the same; The classification reliability of the decision tree in the verification phase is determined based on the number of correct verifications and the total number of verification entities in the decision tree.
6. The method according to any one of claims 1 to 5, characterized in that, The step of obtaining the voting weight of the decision tree for the entity to be classified based on the classification reliability of the decision tree and the node purity of the leaf nodes includes: The voting weights of the decision tree are determined based on the classification reliability of the decision tree, the node purity of the leaf nodes, and the stability coefficient; the stability coefficient is used to balance the classification reliability of the decision tree with the node purity of the leaf nodes of the decision tree.
7. The method as described in claim 6, characterized in that, The process of determining the classification label of the entity to be classified based on the candidate classification labels output by each decision tree and the voting weight of each decision tree for the entity to be classified includes: The voting weights of multiple decision trees that output the same candidate classification label are summed to obtain the voting score; The candidate category label with the highest voting score is used as the category label for the entity to be classified.
8. A computer device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps of the method according to any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method according to any one of claims 1 to 7.
10. A computer program product, characterized in that, The computer program product includes a computer program stored on a computer-readable storage medium, the computer program including program instructions that, when executed by a computer device, cause the computer device to perform the steps of the method as described in any one of claims 1 to 7.