Method for explaining boosting tree by feature attribution based on shp theory

Abstract

The invention discloses a method for explaining a boosting tree by feature attribution based on a shp theory. The method comprises the following steps: S1) preprocessing data; s2) performing model training; s3, calculating a shop value; s4) feature attribution; s5), carrying out visualization; s6) performing model interpretation; and S7) performing model debugging and optimization: performing debugging and optimization on the lifting tree model according to model interpretation and feature attribution results. The invention provides a visual interpretation method, so that a user can understand a decision process and a prediction result of a boosting tree model; feature attribution is realized, and the contribution degree of each feature to a prediction result is accurately determined; a basis is provided for model debugging and optimization, a user is helped to better adjust model parameters, and the performance and accuracy of the model are improved.

Description

Technical Field

[0001] The present invention belongs to the field of credit technology, and in particular relates to a method for feature attribution interpretation boosting tree based on SHAP theory. Background Art

[0002] Boosted tree models, such as XGBoost and LightGBM, have demonstrated powerful predictive capabilities in many fields. However, these complex models are often viewed as black boxes, making their internal decision-making processes difficult to understand. This poses several limitations and risks to their application:

[0003] Lack of explainability: Licensed financial decision-making requires high transparency. Simply knowing the model's prediction results is not enough; we also need to understand how the model makes decisions and which features play a key role in the prediction results. The complex structure of the boosted tree model makes it difficult to intuitively explain the contribution of each feature.

[0004] Difficulty diagnosing problems: When a model makes incorrect predictions or performs poorly, the lack of interpretability makes it difficult to identify the problem and make effective adjustments and improvements. Summary of the Invention

[0005] In order to solve the above technical problems, a technical solution adopted by the present invention is:

[0006] A method for interpreting a boosted tree based on feature attribution in SHAP theory includes the following steps:

[0007] S1): Data preprocessing: Preprocess the internal data involved in the model to improve the quality and availability of the data;

[0008] The data preprocessing specifically includes the following steps:

[0009] S11): Cleaning of format and content errors;

[0010] S12): Logic error cleaning;

[0011] S13): Outlier processing;

[0012] S14): normalization;

[0013] S2): Model training: Use the boosting tree algorithm to train the preprocessed data to obtain a boosting tree model for identifying high-risk customer groups;

[0014] S3): Shap value calculation: use the Shap value theory to calculate the Shap value of each feature in each sample;

[0015] S4): Feature attribution: Determine the contribution of each feature to the prediction result based on the calculated shap value;

[0016] S5): Visualization: Visualize the feature attribution results through the feature influence diagram;

[0017] S6): Model interpretation: By analyzing the shap value distribution of each feature and the interaction between features, the decision-making mechanism of the model is demonstrated;

[0018] S7): Model debugging and optimization: Debug and optimize the boosted tree model based on the model interpretation and feature attribution results.

[0019] Furthermore, in step 4,

[0020] When the feature shap value is greater than 0, the feature improves the prediction value;

[0021] When the feature shape value is less than 0, the feature reduces the prediction value.

[0022] Furthermore, in step 5, the horizontal axis of the feature influence diagram is the shape value, and the vertical axis is the feature parameter. When the feature shape value is positive, it is red, and when the feature shape value is negative, it is blue.

[0023] Furthermore, in step 7, when the contribution of the input variables deviates from conventional knowledge, the performance and accuracy of the model can be improved by adjusting the parameters of the constructed boosting tree model, eliminating controversial variables and tree models, and selecting different feature combinations.

[0024] Furthermore, the shap value calculation in step 3 is calculated using a third-party library function.

[0025] Furthermore, the third-party library function is the shap library in Python.

[0026] Beneficial effects of the present invention:

[0027] This invention enables users to understand the decision-making process and prediction results of the boosted tree model; realizes feature attribution and accurately determines the contribution of each feature to the prediction result; provides a basis for model debugging and optimization, and helps users better adjust model parameters to improve model performance and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 It is a schematic diagram of the system provided by the present invention; DETAILED DESCRIPTION

[0030] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more precise definition of the protection scope of the present invention. Specific embodiment 1:

[0032] like Figure 1 As shown, a method for feature attribution interpretation boosting tree based on SHAP theory includes the following steps:

[0033] S1): Data preprocessing: Preprocess the internal data involved in the model to improve the quality and availability of the data;

[0034] The data preprocessing specifically includes the following steps:

[0035] S11): Cleaning of format and content errors;

[0036] S12): Logic error cleaning;

[0037] S13): Outlier processing;

[0038] S14): normalization;

[0039] S2): Model training: Use the boosting tree algorithm to train the preprocessed data to obtain a boosting tree model for identifying high-risk customer groups;

[0040] S3): SHAP value calculation: SHAP value theory is used to calculate the SHAP value of each feature in each sample. Specifically, the SHAP value calculation in step 3 is performed using a third-party library function. The third-party library function is the SHAP library in Python.

[0041] An example is:

[0042] explainer=shap.TreeExplainer(model);

[0043] shap_values=explainer.shap_values(X),

[0044] Where model is the model trained in step 2, and X is the sample;

[0045] S4): Feature attribution: Determine the contribution of each feature to the prediction result based on the calculated shap value; specifically, in step 4,

[0046] When the feature shap value is greater than 0, the feature improves the prediction value;

[0047] When the feature shape value is less than 0, the feature reduces the prediction value.

[0048] S5): Visualization: The feature attribution results are visualized through the feature influence diagram; specifically, in step 5, the horizontal axis of the feature influence diagram is the shap value, and the vertical axis is the feature parameter. When the feature shap value is positive, it is red, and when the feature shap value is negative, it is blue.

[0049] S6): Model interpretation: By analyzing the shap value distribution of each feature and the interaction between features, the decision-making mechanism of the model is demonstrated;

[0050] S7): Model debugging and optimization: Based on the model interpretation and feature attribution results, the boosted tree model is debugged and optimized. Specifically, in step 7, if the contribution of the input variables deviates from conventional understanding, the performance and accuracy of the model can be improved by adjusting the parameters of the constructed boosted tree model, removing controversial variables and tree models, and selecting different feature combinations.

[0051] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's description and drawings, or directly or indirectly applied to other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for feature attribution explanation boosting tree based on SHAP theory, characterized by: The following steps are involved: S1): Data preprocessing: preprocess the internal data involved in the model to improve the quality and availability of the data; The data preprocessing specifically includes the following steps: S11): format content error cleaning; S12): logical error cleaning; S13): outlier processing; S14): Normalization; S2): Model training: Use the boosting tree algorithm to train the preprocessed data to obtain a boosting tree model for identifying high-risk customer groups; S3): Shap value calculation: use the Shap value theory to calculate the Shap value of each feature in each sample; S4): Feature attribution: According to the calculated shap value, determine the contribution of each feature to the prediction result; S5): Visualization: The feature attribution results are visualized through the feature influence diagram; S6): Model explanation: By analyzing the shap value distribution of each feature and the interaction between features, the decision-making mechanism between models is demonstrated; S7): Model debugging and optimization: Debug and optimize the boosted tree model based on the model interpretation and feature attribution results.

2. According to the method of claim 1, the feature attribution explanation boosting tree based on SHAP theory is characterized by: In step 4, When the feature shap value is greater than 0, the feature improves the prediction value; When the feature shap value is less than 0, the feature reduces the prediction value.

3. The method of feature attribution explanation boosting tree based on SHAP theory according to claim 1 is characterized by: In step 5, the horizontal axis of the feature influence diagram is the shap value, and the vertical axis is the feature parameter. When the feature shap value is positive, it is red, and when the feature shap value is negative, it is blue.

4. The method of feature attribution explanation boosting tree based on SHAP theory according to claim 1, characterized in that: In step 7, when the contribution of the input variables deviates from the conventional understanding, the performance and accuracy of the model can be improved by adjusting the parameters of the constructed boosting tree model, eliminating controversial variables and tree models, and selecting different feature combinations.

5. The method of feature attribution explanation boosting tree based on SHAP theory according to claim 1, characterized in that: The shap value calculation in step 3 is calculated using a third-party library function.

6. The method of feature attribution explanation boosting tree based on SHAP theory according to claim 1, characterized in that: The third-party library function is the shap library in Python.

Images