A machine learning classification model explanation method based on a greedy algorithm

Abstract

A machine learning classification model explanation method based on a greedy algorithm, the method uses a greedy algorithm to combine feature boxes, and obtains feature box combinations respectively starting from each screened box. The feature box combinations learned by feature matching can have a more comprehensive understanding of the feature statistics of the target and make explanations for the machine learning results. This method uses a greedy strategy to screen and combine features, providing a more intuitive customer screening method for enterprises. In practical applications, it is mainly used to assist in explaining the prediction results of black box models and serve front-line business personnel.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence and data processing, and more particularly relates to a machine learning classification model explanation method based on a greedy algorithm. BACKGROUND

[0002] Machine learning, as one of the most critical technologies of artificial intelligence, is often considered by the outside world as the basis of AI application. Algorithm engineers believe that algorithm capabilities are directly applied to financial, industrial, pharmaceutical, and Internet industries with good digital foundation, providing intelligent risk control, predictive maintenance, drug discovery, personalized recommendation, and other services for enterprises. However, this obviously does not take into account that many industries need to obtain relatively transparent algorithm processes in order to accurately interpret the results. Especially for machine learning models, although the accuracy of the model has approached 100%, it is still unable to clearly explain whether the model has reasonably classified, and can only choose to believe the results predicted by the model. Under such a premise, in order to realize an algorithm explanation applicable to front-line scenarios, the present application sets forth a machine learning classification model explanation method based on a greedy algorithm. SUMMARY

[0003] In view of the deficiencies of the prior art, the present application provides a machine learning classification model explanation method based on a greedy algorithm, which provides a more realistic result interpretation for business personnel of an enterprise when facing target customer prediction results. The purpose is to assist business personnel in rapid auditing, increase performance, and improve service quality and customer satisfaction.

[0004] The purpose of the present application is achieved by the following technical solutions:

[0005] A machine learning classification model explanation method based on a greedy algorithm,

[0006] Step one: obtaining customer historical feature data, behavior data, and classification labels in a training set;

[0007] Step two: cleaning the customer historical feature data and behavior data, and then retaining the feature with the highest information value among the features with a correlation reaching a threshold value;

[0008] Step three: performing binning processing on the features processed in step two to form a feature table for inputting a machine learning model;

[0009] Step four: training a machine learning classification model with structured data input using the feature table and the classification labels to obtain a feature importance ranking of the machine learning classification model;

[0010] Step five: calculate the information value of each box and the target customer number of the box, combine the feature importance ranking to build the box index of each box; retain the box whose box index is greater than the box screening threshold as the feature box;

[0011] Step six: take each feature box as the starting point, find a series of feature box combinations for target customers through the greedy algorithm, as the feature box combination dictionary;

[0012] Step seven: input the new customer feature data and behavior data after preprocessing in step two and binning processing in step three into the trained machine learning classification model to obtain the predicted label of the new customer; if the new customer is judged to be a target customer according to the predicted label, the feature of the new customer matches the feature box combination dictionary, and the hit feature box combination is output as the explanation of the machine learning classification model.

[0013] Further, the cleaning of the customer historical feature data and behavior data specifically includes dirty data cleaning, missing value processing, and repeated value deletion.

[0014] Further, after cleaning the customer historical feature data and behavior data in step two, the non-numeric category features also need to be encoded.

[0015] Further, in step three, the category type feature is one box, the date type feature is binned by year or month, and the numerical type feature is binned by the minimum conditional entropy binning method;

[0016] The minimum conditional entropy binning method for binning processing specifically includes:

[0017] (1) Take each value of the feature as a division point in turn, divide the feature into two, and calculate the conditional entropy weighted sum of the two boxes of the feature:

[0018]

[0019] where H(Y) represents the conditional entropy value weighted sum of the feature, represents the box number proportion, p j is the customer proportion or non-customer proportion in a single box x i

[0020] (2) Select the value with the minimum conditional entropy weighted sum as the division point to divide one box into two boxes;

[0021] (3) Select the box with the maximum conditional entropy in the feature, repeat steps (1) and (2), and continue to divide the box until the number of bins of the feature reaches the set upper limit of the number of bins of a single feature. ​

[0022] Further, the calculation formula of the box index of each box in step five is as follows:

[0023]

[0024] Wherein, S represents the box index, p vi represents the application ratio of information value, p target represents the application ratio of box quantity proportion, p feature_importance represents the application ratio of feature importance proportion; IV i represents the information value of the i-th box; IV max represents the maximum value of information value in all boxes; Target i represents the number of target customers of the i-th box; Target max represents the maximum value of the number of target customers in all boxes; Feature_importance I represents the feature importance of the i-th feature obtained by the machine learning model; Feature_importance MAX represents the maximum feature importance obtained by the machine learning model.

[0025] Further, in step six, a series of feature box combinations for target customers are found by taking each feature box as a starting point through a greedy algorithm, which specifically includes:

[0026] (1) Taking a single feature box as a starting point, combining the feature box with other feature boxes one by one, and screening out a feature two-box combination with the highest probability of target customers;

[0027] (2) Taking the two-box combination as a starting point, combining the two-box combination with other feature boxes one by one, and screening out a feature three-box combination with the highest probability of target customers, and setting an upper limit on the number of boxes in the feature binning combination to stop the loop;

[0028] (3) replacing the starting feature box, repeating steps (1) to (2), and retaining all feature box combinations to form a feature box combination dictionary.

[0029] Further, if the classification labels of customers in the training set have more than two categories, the classification labels are simplified into binary classification, where 1 is the label of target customers and 0 is the label of non-target customers.

[0030] Further, in step two, Pearson correlation analysis is used to calculate the correlation of features.

[0031] Further, one-hot encoding is used for feature encoding of non-numeric category features.

[0032] Further, since the one-hot code will increase the feature dimension, the non-numeric category features are encoded by using the one-hot code, and then the principal component analysis method is used to reduce the feature dimension according to the expert experience.

[0033] The beneficial effects of the present application are as follows:

[0034] The present application adopts the minimum conditional entropy binning method to categorize all features, and then according to the feature importance given by the machine learning classification model, the information value of the feature and the target quantity are used as screening indexes to screen the feature bins with high target probability, and then a greedy algorithm is used to combine the feature bins. The above technical means are all for summarizing the feature of the data set to the greatest extent, reducing the feature dimension, so as to statistically explain the result of the machine learning classification model. The explanation method not only conforms to the result of the classification model, but also does not need professional knowledge to interpret, widens the use scene of the machine learning classification model, speeds up the application deployment of the model, reduces the understanding threshold of the model for the first-line business personnel, helps the general business personnel to manage customers, gives a more real result interpretation, and reduces customer complaint problems.

[0035] Secondly, the effect of the present application is applicable to all machine learning classification models. On the market, a new model is generally used to explain the feature contribution of some machine learning models, which not only increases the difficulty of explanation, but also the results of the two models cannot completely conform. The present method does not involve the mathematical method interpretation of the machine learning classification model, but only analyzes the initial data set to find the explanation conforming to the result, reduces the interpretation difficulty of the machine learning classification model, and has a more extensive application range. BRIEF DESCRIPTION OF DRAWINGS

[0036] Hereinafter, exemplary embodiments will be described with reference to the accompanying drawings, in which exemplary embodiments are illustrated. The following description, taken in conjunction with the accompanying drawings, is presented to illustrate the principles of the present application. It will be apparent to those skilled in the art that the present application is not limited to the embodiments described in the following detailed description. Rather, the present application includes all alternatives, modifications and equivalents falling within the scope of the present application as detailed in the claims.

[0037] The block diagrams shown in the accompanying drawings are merely functional entities, and do not necessarily correspond to physically independent entities. That is, the functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0038] The flowcharts shown in the drawings are merely illustrative and are not necessarily required to include all the contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be further divided, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to actual conditions.

[0039] Figure 1 The method for explaining a machine learning classification model based on a greedy algorithm for an embodiment of the application. DETAILED DESCRIPTION

[0040] The exemplary embodiments will be described in detail herein with reference to the accompanying drawings. In the following description, unless otherwise indicated, like numbers in the different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments are not meant to represent all implementations consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0041] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0042] It will be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order or hierarchy. These terms are used only to distinguish one from another. For example, a first information can be termed a second information, and, similarly, a second information can be termed a first information, without departing from the scope of the present application. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "in response to determining" or "in response to a determination."

[0043] The present embodiment aims to assist in explaining the results of a complex machine learning model through a greedy algorithm, to achieve good application of real business, and to evaluate the risk of loan customers. The risk monitoring of bank loans can be divided into three stages: before lending, during lending, and after lending. The present method can be implemented in these three stages, the main difference being the range of customer data, which provides a basis for risk assessment for banks.

[0044] The specific content of the method described in the present embodiment takes the post-lending stage as an example, including the following steps:

[0045] Step one: Obtain the historical feature data, behavior data, and classification labels of customers in the training set.

[0046] The bank data features are large in quantity, and under the traditional management architecture, the information and data of each department do not circulate, the collaborative efficiency is not high, and the data quality is uneven, resulting in high data sorting cost, low efficiency, and slow practicality. Therefore, personnel from multiple departments need to assist in sorting the data framework, extracting and sorting the data. Obtain customer feature data, behavior data, and five-level classification labels. The customer feature data includes customer personal information, loan data, and credit data.

[0047] The customer behavior data includes customer loan behavior, deposit behavior, and transfer behavior. The five-level classification labels are five levels of classification of commercial banks according to the actual repayment ability of borrowers, which are normal, attention, substandard, suspicious, and loss. Substandard and below loans are non-performing loans. Form a data set D:

[0048]

[0049] In the data set D, x nm represents the mth feature of the nth customer, y n represents the label of the nth customer, wherein y i ∈(“normal”, “attention”, “substandard”, “suspect”, “loss”).

[0050] Step two: clean the customer historical feature data and behavior data, and then retain the information value of the highest feature of the features with a correlation reaching a threshold.

[0051] Specifically, cleaning the customer historical feature data and behavior data includes dirty data cleaning, missing value processing, and duplicate value deletion. Dirty data cleaning includes deleting incorrect ID numbers, data expressions with the same meaning “unknown”, and any data that does not meet the field specification length or format; the missing value processing method includes filling 0 values or mean values or median values or mode values.

[0052] Non-numeric category features also need to be encoded, and the feature encoding method selects the label encoding method, such as one-hot code, which can convert non-numeric category fields into numeric category fields, facilitating model training. However, due to the increase in feature dimension caused by one-hot code, the principal component analysis method is used to reduce the feature dimension after one-hot code is used to encode the non-numeric category features.

[0053] When there are many features, features with poor data quality can also be removed, and the removal conditions include a field null rate of 90% or more and a single category number of 90% or more.

[0054] If the classification labels of the customers in the training set have more than two categories, the classification labels are simplified into binary classification, where 1 is the label of the target customer and 0 is the label of the non-target customer.

[0055] In this embodiment, the label data preprocessing is divided into two parts, the first is to simplify the five-level classification into two-level classification, and the normal and attention loan customers are classified into one class, and are determined as good customers, and the substandard, doubtful and loss loan customers are classified into one class, and are determined as bad customers. i ∈(0,1,y i = 1 represents a bad sample, i.e. a default sample; y i = 0 represents a good sample.

[0056] Step three: the features processed in step two are binned to form a feature table for the final input machine learning model.

[0057] Among them, a category type feature is a box, a date type feature is binned by year or month, and a numerical type feature is binned by the minimum conditional entropy binning method.

[0058] The minimum conditional entropy binning is a top-down binning method. Binning by the minimum conditional entropy binning method includes the following steps:

[0059] (1) Take each value of the feature as a division point in turn, divide the data set into two, and calculate the weighted sum of the conditional entropy of the two boxes of the feature:

[0060]

[0061] where H(Y) represents the weighted sum of the conditional entropy of the feature, represents the proportion of the number of boxes, p j is the proportion of customers or non-customers in a single box x i .

[0062] (2) Select the value with the minimum weighted sum of conditional entropy as the division point, and divide one box into two boxes;

[0063] (3) Select the box with the maximum conditional entropy in the feature, repeat steps (1) and (2), and continue to divide the box until the number of bins of the feature reaches the set upper limit of the number of bins of a single feature.

[0064] The advantage of using the minimum conditional entropy binning method is that it can get purer bins, i.e. the proportion of target customers / non-target customers in the bins is maximized. If the proportion of each category of the dependent variable in the i-th bin is equal, i.e. i , then the entropy value of the i-th bin reaches the maximum value; if the dependent variable in the i-th bin has only one value, i.e. some p

[0065] The category type feature is usually not binned, and one category is one bin. If there are too many categories, the small number of categories can be combined according to the situation. The binning of the date type feature is based on the distribution graph of the feature. Whether there is a significant periodic feature is observed, and the feature is binned according to the period length.

[0066] Step four: training a machine learning classification model with structured data as input using the feature table and classification label to obtain the feature importance ranking of the machine learning classification model.

[0067] In this embodiment, a decision tree model is used to predict the loan default rate of customers. The sampling strategy is random down sampling, and the ratio of default customers to non-default customers is 2:1. The ratio of training set, validation set and test set is 6:2:2. The feature_importances_(model feature importance) ranking is obtained.

[0068] Step five: calculate the information value of each bin and the target customer number of the bin, combine the feature importance ranking, and construct the bin index of each bin; keep the bin whose bin index is greater than the bin screening threshold as the feature bin.

[0069] The information value is an index used to represent the contribution of the feature to the target prediction, that is, the classification prediction ability of the feature. Generally, the higher the information value, the stronger the prediction ability of the feature, because the higher the information contribution. The condition for using information value is that the task must be a supervised learning task, that is, there is a classification label. Secondly, the task must be a binary classification task, which is reflected in the specific formula:

[0070]

[0071] where IV i represents the information value of a certain bin; T i is the number of target samples in the bin, that is, the number of customers; NT i is the number of non-target samples in the bin, that is, the number of samples; T t is the number of all target samples in the feature; NT t is the number of all non-target samples in the feature.

[0072] In this embodiment, the IV value of the bin is calculated, and the specific formula is as follows:

[0073]

[0074] where IV i represents the IV value of a certain bin; Bad i is the number of bad samples in the bin, that is, the number of default customers; Good iGood t Bad t Good

[0075] Information value, the number of target customers of the box, and feature importance ranking are three different dimensions of indicators. In order to normalize the three indicators, the following formula is used to construct the box indicator of each box:

[0076]

[0077] Where S represents the box indicator, p vi represents the application proportion of information value, p target represents the application proportion of box quantity ratio, p feature_importance represents the application proportion of feature importance ratio; IV i represents the information value of the i-th box; IV max represents the maximum value of information value in all boxes; Target i represents the number of target customers of the i-th box; Target max represents the maximum value of target customer number in all boxes; Feature_importance I represents the feature importance of the i-th feature obtained by the machine learning model; Feature_importance MAX represents the maximum feature importance obtained by the machine learning model.

[0078] If the box indicator S is greater than the threshold value, the box is retained, otherwise it is rejected. The reason for this is that first, the minimum conditional entropy value binning will obtain a part of the box with a larger proportion of good samples, which is not what we want; second, most banks have a large number of customers, and a simplified method is needed.

[0079] Step six: taking each feature box as a starting point, a series of feature box combinations for target customers are found through a greedy algorithm, which are used as the feature box combination dictionary. The specific steps include the following sub-steps:

[0080] (1) Taking a single feature box as a starting point, the feature box is combined with other feature boxes one by one, and the feature two-box combination with the largest probability of target customers is selected;

[0081] (2) Taking the two-box combination as a starting point, the two-box combination is combined with other feature boxes one by one, and the feature three-box combination with the largest probability of target customers is selected, and the upper limit of the number of boxes in the feature binning combination is set to stop the loop;

[0082] (3) replace the starting feature box, repeat steps (1) ~ (2), keep all the combination of feature box, form the feature box combination dictionary.

[0083] For example:

[0084] (1) calculate the default probability of the first feature box a.

[0085] (2) calculate the default probability of feature box a and other single box (b, c, d……) respectively, select the combination of two boxes with the largest default probability (such as the combination of box a and b), compare the default probability of the box combination with the default probability of box a, if greater than the default probability of box a, continue to find the next box (c, d……), form a three-box combination; otherwise stop. Repeat this step until the default probability of the box combination starting with box a no longer increases or the number of boxes in the box combination reaches the preset upper limit. Input the feature box combination result into the high default customer feature box combination dictionary.

[0086] (3) replace the next feature box b as the starting point, repeat the above (1) and (2) steps, until all feature boxes are used as starting points, then get the required high default customer feature box combination dictionary.

[0087] Because the combination result of the greedy algorithm has the characteristics of local optimization, it is necessary to find all the feature combinations with the highest default rate with all the filtered features as the starting point, to reduce the disturbance caused by local optimization.

[0088] Step seven: input the new customer feature data and behavior data after preprocessing in step two and box processing in step three into the trained machine learning classification model to get the prediction label of the new customer; if the new customer is judged to be the target customer according to the prediction label, the feature of the new customer matches the feature box combination dictionary, and the hit feature box combination is output as the explanation of the machine learning classification model. Thus, it assists the front-line business personnel to judge the customer risk.

[0089] Those skilled in the art can understand that the above description is only a preferred example of the application and is not used to limit the application, although the application has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features. Any modification, equivalent replacement, etc. within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. A method for explaining a machine learning classification model based on a greedy algorithm, characterized in that: Step 1: Obtain customer historical feature data, behavior data, and classification labels in the training set; Step 2: Clean the customer historical feature data and behavior data, and then retain the feature with the highest information value that meets the correlation threshold; Step 3: Perform binning processing on the features processed in Step 2 to form a feature table for inputting into the machine learning model; Step 4: Train a machine learning classification model with structured data using the feature table and classification labels to obtain a feature importance ranking of the machine learning classification model; Step 5: Calculate the information value of each bin and the number of target customers in the bin, combine the feature importance ranking, and construct a bin index for each bin; retain the bins with a bin index greater than the bin screening threshold as feature bins; Step 6: Start with each feature bin, find a series of feature bin combinations for target customers through a greedy algorithm, and use the feature bin combinations as a dictionary; Step 7: After preprocessing the new customer feature data and behavior data in Step 2 and binning processing in Step 3, input them into the trained machine learning classification model to obtain the predicted label of the new customer; if the new customer is determined to be a target customer according to the predicted label, match the feature of the new customer with the feature bin combination dictionary, and output the hit feature bin combination as an explanation of the machine learning classification model; In Step 3, category features are one bin, date features are binned by year or month, and numerical features are binned using the minimum conditional entropy binning method; wherein The minimum conditional entropy binning method includes: (1) Take each value of the feature as a division point to divide the feature into two parts, and calculate the weighted sum of the conditional entropy of the two bins: wherein H(Y) represents the characteristic conditional entropy value weighted sum, represents the proportion of the number of boxes, p j for a single box x i proportion of customers or non-customers; (2) Select the value with the minimum weighted sum of conditional entropy as the division point to divide one bin into two bins; (3) Select the bin with the maximum conditional entropy in the feature, repeat steps (1) and (2), and continue to divide the bin until the number of bins for a single feature reaches the upper limit of the number of bins for a single feature. 2.The greedy algorithm based machine learning classification model explanation method of claim 1, wherein, The cleaning of the customer historical feature data and behavior data includes dirty data cleaning, missing value processing, and duplicate value deletion. 3.The greedy algorithm based machine learning classification model explanation method of claim 1, wherein, After cleaning the customer historical feature data and behavior data in Step 2, the category features that are not numerical also need to be encoded. 4.The greedy algorithm based machine learning classification model explanation method of claim 1, wherein, The calculation formula of the bin index of each bin in Step 5 is as follows: wherein S represents the box index, p vi represents the application ratio of information value, p target represents the application ratio of box quantity proportion, p feature_importance represents the application ratio of feature importance proportion; IV i represents the information value of the i-th box; IV max represents the maximum value of information value in all boxes; Target i represents the number of target customers of the i-th box; Target max represents the maximum value of the number of target customers in all boxes of the index; Feature_importance I represents the feature importance of the i-th feature obtained by the machine learning model; Feature_importance MAX represents the maximum feature importance obtained by the machine learning model. 5.The greedy algorithm based machine learning classification model explanation method of claim 1, wherein, In Step 6, starting with each feature bin, a series of feature bin combinations for target customers are found through a greedy algorithm, which includes: (1) Start with a single feature bin, combine it with other feature bins one by one, and select the feature two-bin combination with the highest probability of target customers; (2) Start with the two-bin combination, combine it with other feature bins one by one, and select the feature three-bin combination with the highest probability of target customers; set an upper limit for the number of bins in the feature bin combination to stop the loop. (3) replace the initial feature box, repeat steps (1) ~ (2), keep all the combination of feature box, constitute the feature box combination dictionary.

6. The greedy algorithm based machine learning classification model explanation method of claim 1, wherein, If the classification label of the customer in the training set has more than two categories, the classification label is simplified into a binary classification, where 1 is the label of the target customer and 0 is the label of the non-target customer.

7. The greedy algorithm based machine learning classification model explanation method of claim 1, wherein, In step two, the Pearson correlation analysis is used to calculate the correlation of the features.

8. The greedy algorithm based machine learning classification model explanation method of claim 3, wherein, The non-numeric category features are encoded by using the one-hot code.

9. The greedy algorithm based machine learning classification model explanation method of claim 8, wherein, Since the one-hot code will increase the feature dimension, after the non-numeric category features are encoded by using the one-hot code, the principal component analysis method is used to reduce the feature dimension according to the expert experience.

Images